2024-09-12T01:57:21.575 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T01:57:21.601 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T01:57:21.696 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901597' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGa9HOdbvyYqe8Kkb2ASQ7Gu1O6PsJCo4JH3pQmLApvZRHPUV6Be8pkdEdjMpiIk+LYOzjBK0xhfkpCANcrk+EQ= smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4tJOGiDpUheyQkAnERg9Ogr3XaNedPNE6vpzc0mReUl8laXRiCM8A0X+xbEW0meh5ewUQ0J763g9Z0SSqJ874= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOL9FETrz0p+zhEESSZTaQ8zjVrdnR3w6FNfB8kxcR+7tCFi9thn81y0eI8veG7aIR6hqav96qthMY/8dczWbMM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T01:57:21.697 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T01:57:21.697 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T01:57:21.697 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T01:57:21.699 INFO:teuthology.task.internal:Checking packages... 2024-09-12T01:57:21.721 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T01:57:21.721 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T01:57:21.721 INFO:teuthology.packaging:ref: None 2024-09-12T01:57:21.721 INFO:teuthology.packaging:tag: None 2024-09-12T01:57:21.721 INFO:teuthology.packaging:branch: squid 2024-09-12T01:57:21.721 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T01:57:21.721 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T01:57:21.966 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T01:57:21.968 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T01:57:21.973 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T01:57:21.973 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T01:57:21.981 INFO:teuthology.task.internal:Saving configuration 2024-09-12T01:57:22.000 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T01:57:22.006 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T01:57:22.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:52:29.911842', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGa9HOdbvyYqe8Kkb2ASQ7Gu1O6PsJCo4JH3pQmLApvZRHPUV6Be8pkdEdjMpiIk+LYOzjBK0xhfkpCANcrk+EQ='} 2024-09-12T01:57:22.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:52:29.914241', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM4tJOGiDpUheyQkAnERg9Ogr3XaNedPNE6vpzc0mReUl8laXRiCM8A0X+xbEW0meh5ewUQ0J763g9Z0SSqJ874='} 2024-09-12T01:57:22.098 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 01:52:29.913129', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOL9FETrz0p+zhEESSZTaQ8zjVrdnR3w6FNfB8kxcR+7tCFi9thn81y0eI8veG7aIR6hqav96qthMY/8dczWbMM='} 2024-09-12T01:57:22.098 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T01:57:22.103 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T01:57:22.104 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T01:57:22.104 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T01:57:22.104 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T01:57:22.221 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe18e933490>, signals=[15]) 2024-09-12T01:57:22.221 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T01:57:22.227 INFO:teuthology.task.internal:Opening connections... 2024-09-12T01:57:22.227 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T01:57:22.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:57:22.326 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-09-12T01:57:22.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:57:22.410 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-09-12T01:57:22.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:57:22.492 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T01:57:22.498 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-09-12T01:57:22.516 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-09-12T01:57:22.516 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-09-12T01:57:22.573 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-09-12T01:57:22.573 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:57:22.574 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:57:22.575 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-09-12T01:57:22.607 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-09-12T01:57:22.625 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-09-12T01:57:22.625 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-09-12T01:57:22.683 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-09-12T01:57:22.683 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-09-12T01:57:22.683 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-09-12T01:57:22.683 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:57:22.683 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:57:22.684 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:57:22.685 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-09-12T01:57:22.715 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-09-12T01:57:22.732 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-09-12T01:57:22.733 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T01:57:22.789 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T01:57:22.790 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T01:57:22.790 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-09-12T01:57:22.815 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T01:57:22.824 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T01:57:22.829 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T01:57:22.830 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:57:22.832 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:57:22.835 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-09-12T01:57:22.851 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T01:57:22.856 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T01:57:22.856 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:57:22.889 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:57:22.892 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-09-12T01:57:22.916 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T01:57:23.019 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T01:57:23.034 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T01:57:23.034 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T01:57:23.034 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T01:57:23.035 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T01:57:23.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T01:57:23.052 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-12T01:57:23.110 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-503.el9.x86_64 2024-09-12T01:57:23.110 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-503.el9.x86_64 2024-09-12T01:57:23.111 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-09-12T01:57:25.622 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-12T01:57:27.639 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-12T01:57:32.240 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-12T01:57:34.182 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-12T01:57:34.961 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-12T01:57:40.608 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T01:57:42.732 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T01:57:42.732 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T01:57:42.817 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T01:57:42.824 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-09-12T01:57:42.825 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:57:42.826 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 79 M 2024-09-12T01:57:42.826 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 126 M 2024-09-12T01:57:42.826 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-12T01:57:43.376 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 77 kB/s | 23 kB 00:00 2024-09-12T01:57:59.558 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 972 kB/s | 16 MB 00:16 2024-09-12T01:58:02.991 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 1.5 MB/s | 29 MB 00:19 2024-09-12T01:58:05.491 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 1.5 MB/s | 35 MB 00:22 2024-09-12T01:58:05.491 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-12T01:58:05.492 INFO:teuthology.orchestra.run.smithi003.stdout:Total 3.5 MB/s | 79 MB 00:22 2024-09-12T01:58:06.203 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-12T01:58:06.569 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-12T01:58:06.569 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-12T01:58:07.542 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-12T01:58:07.542 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-12T01:58:10.129 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-12T01:58:10.803 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:58:10.894 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:58:13.670 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:58:13.804 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:58:18.926 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:58:18.980 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:58:24.117 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:59:02.724 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:59:02.765 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:59:03.156 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:59:03.156 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T01:59:03.156 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T01:59:03.156 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.004 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:59:04.005 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-12T01:59:04.351 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:59:04.444 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.444 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:59:04.445 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T01:59:04.445 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T01:59:04.445 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T01:59:04.445 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T01:59:04.445 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T01:59:04.445 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T01:59:04.502 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T01:59:04.503 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T01:59:04.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-09-12T01:59:04.536 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-09-12T01:59:04.536 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-09-12T01:59:04.536 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 82.el9 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8274670 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T01:59:04.537 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T01:59:04.538 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T01:59:04.539 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T01:59:04.539 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-505.el9.x86_64 2024-09-12T01:59:04.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T01:59:05.210 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-09-12T01:59:06.456 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T01:59:06.486 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-09-12T01:59:06.488 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-09-12T01:59:06.503 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.MGLJUQGbnm 2024-09-12T01:59:06.504 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MGLJUQGbnm 2024-09-12T01:59:06.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.MGLJUQGbnm 2024-09-12T01:59:06.717 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.MGLJUQGbnm is 6KB 2024-09-12T01:59:06.766 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.MGLJUQGbnm 2024-09-12T01:59:06.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T01:59:06.844 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T01:59:06.844 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T01:59:06.844 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T01:59:06.844 INFO:teuthology.orchestra.run.smithi003.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T01:59:06.845 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T01:59:06.997 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-09-12T01:59:37.000 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T01:59:37.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T01:59:37.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T01:59:55.453 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-09-12T02:00:04.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T02:00:04.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:01:04.465 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T02:01:16.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T02:01:16.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:01:16.764 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-12T02:01:17.119 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T02:01:17.119 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:01:17.119 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-09-12T02:01:17.136 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:01:17.137 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:01:17.137 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:01:17.137 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:01:18.138 DEBUG:teuthology.parallel:result is None 2024-09-12T02:01:18.138 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T02:01:18.145 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T02:01:18.146 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:01:18.148 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:01:18.150 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:01:18.169 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T02:01:18.215 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T02:01:18.222 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T02:01:18.222 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:01:18.224 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:01:18.227 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:01:18.260 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T02:01:18.266 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T02:01:18.266 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:01:18.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:18.309 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:01:18.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:18.327 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:01:18.330 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:01:18.352 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:01:18.386 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.405 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.410 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.426 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.427 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.441 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:01:18.442 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T02:01:18.449 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T02:01:18.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:01:18.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:01:18.454 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:01:18.481 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T02:01:18.490 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T02:01:18.490 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:01:18.521 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:01:18.523 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:01:18.540 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:18.628 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:18.670 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:18.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:18.796 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:01:18.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:01:18.863 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:18.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:18.957 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:19.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:19.083 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:01:19.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:01:19.151 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:19.199 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:01:19.258 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:19.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:01:19.346 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:01:19.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:01:19.412 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-09-12T02:01:19.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-09-12T02:01:19.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-09-12T02:01:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:01:19.457 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:01:19.493 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:01:19.820 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T02:01:19.828 INFO:teuthology.task.internal:Starting timer... 2024-09-12T02:01:19.828 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T02:01:19.839 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T02:01:19.847 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T02:01:19.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-09-12T02:01:19.929 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-09-12T02:01:19.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-09-12T02:01:19.980 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-09-12T02:01:19.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-09-12T02:01:20.025 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-09-12T02:01:20.027 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T02:01:20.027 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-09-12T02:01:20.051 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-09-12T02:01:20.051 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-09-12T02:01:20.073 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-09-12T02:01:20.073 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-09-12T02:01:20.094 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-09-12T02:01:20.094 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive'} 2024-09-12T02:01:20.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:01:20.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:20.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:01:20.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:20.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:01:20.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:01:20.181 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T02:01:20.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:01:20.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:01:20.257 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:01:20.289 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T02:01:20.298 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T02:01:20.302 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T02:01:20.487 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T02:01:20.498 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T02:01:20.499 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi026.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T02:05:32.115 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-09-12T02:05:32.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T02:05:32.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:05:32.202 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-09-12T02:05:32.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-09-12T02:05:32.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-12T02:05:32.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:05:32.382 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-12T02:05:32.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-12T02:05:32.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-12T02:05:32.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:05:32.567 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-09-12T02:05:32.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-09-12T02:05:32.646 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T02:05:32.656 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T02:05:32.657 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:05:32.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:05:32.660 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:05:32.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:05:32.662 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:05:32.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:05:32.689 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:05:32.694 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:05:32.705 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:05:32.710 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:05:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:05:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:05:32.732 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-09-12T02:05:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-09-12T02:05:32.744 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-09-12T02:05:32.749 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-09-12T02:05:32.753 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-09-12T02:05:32.760 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:05:32.764 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:05:32.764 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-09-12T02:05:32.775 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:05:32.778 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:05:32.779 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:05:32.792 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:05:32.835 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.839 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:05:32.842 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:05:32.842 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-12T02:05:32.842 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.842 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.842 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.843 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.856 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:05:32.859 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:05:32.860 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-12T02:05:32.860 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.860 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.860 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.860 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:05:32.861 INFO:teuthology.run_tasks:Running task install... 2024-09-12T02:05:32.869 DEBUG:teuthology.task.install:project ceph 2024-09-12T02:05:32.869 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T02:05:32.869 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T02:05:32.869 INFO:teuthology.task.install:Using flavor: default 2024-09-12T02:05:32.875 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T02:05:32.875 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T02:05:32.875 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:05:32.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:05:32.878 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:05:32.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:05:32.879 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:05:32.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:05:33.025 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:05:33.026 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:05:33.041 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:05:33.042 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:05:33.053 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:05:33.053 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:05:33.152 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:05:33.153 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:05:33.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:05:33.163 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:05:33.163 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:05:33.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:05:33.174 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:05:33.174 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:05:33.174 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:05:33.180 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:05:33.180 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:05:33.189 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:05:33.189 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:05:33.200 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:05:33.200 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:05:33.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:05:33.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:05:33.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:05:33.324 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:05:33.332 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:05:33.340 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:05:33.390 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-09-12T02:05:33.391 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-09-12T02:05:33.398 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-09-12T02:05:33.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-09-12T02:05:33.404 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-09-12T02:05:33.406 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-09-12T02:05:33.627 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-09-12T02:05:33.629 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-09-12T02:05:33.630 INFO:teuthology.orchestra.run.smithi026.stdout:45 files removed 2024-09-12T02:05:33.655 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:05:33.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:05:33.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:05:34.217 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T02:05:34.222 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 290 kB/s | 84 kB 00:00 2024-09-12T02:05:34.223 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-12T02:05:34.501 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T02:05:34.515 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T02:05:34.515 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T02:05:34.777 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T02:05:34.782 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-12T02:05:34.791 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T02:05:35.332 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-12T02:05:35.535 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-12T02:05:36.766 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-12T02:05:37.269 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-12T02:05:38.641 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 20 MB 00:02 2024-09-12T02:05:38.851 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-12T02:05:43.279 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-12T02:05:43.335 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-12T02:05:43.725 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-12T02:05:44.772 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-12T02:05:44.819 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 19 kB 00:00 2024-09-12T02:05:45.401 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-12T02:05:45.775 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-12T02:05:45.840 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-12T02:05:46.288 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T02:05:51.664 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-12T02:05:51.720 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T02:05:51.794 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T02:05:54.041 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.042 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.053 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:05:54.054 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.055 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:05:54.056 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.058 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:05:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.061 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.079 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.080 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.091 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:05:54.092 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.093 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:05:54.093 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.095 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:05:54.097 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.098 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.127 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-09-12T02:05:54.145 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-12T02:05:54.145 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:05:54.146 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:05:54.147 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:05:54.148 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:05:54.149 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:05:54.150 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:05:54.151 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:05:54.152 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:05:54.153 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:05:54.154 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:05:54.155 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:05:54.156 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:05:54.157 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:05:54.158 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:====================================================================================== 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:Install 148 Packages 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 8 Packages 2024-09-12T02:05:54.159 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:05:54.160 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.160 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.160 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 226 M 2024-09-12T02:05:54.160 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-09-12T02:05:54.161 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-09-12T02:05:54.174 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:05:54.175 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:05:54.176 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:05:54.177 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:05:54.179 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:05:54.180 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:05:54.181 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-09-12T02:05:54.182 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:05:54.183 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:05:54.184 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:05:54.185 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:05:54.186 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:05:54.187 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:05:54.188 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:05:54.189 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:05:54.190 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:05:54.191 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:05:54.192 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================== 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 8 Packages 2024-09-12T02:05:54.193 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:05:54.194 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.194 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:05:54.194 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 226 M 2024-09-12T02:05:54.194 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-09-12T02:05:54.252 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-09-12T02:05:54.262 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-12T02:05:54.262 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-09-12T02:05:54.262 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:05:54.263 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:05:54.264 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:05:54.265 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:05:54.266 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:05:54.267 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:05:54.268 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:05:54.269 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:05:54.270 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:05:54.271 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:05:54.272 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:05:54.273 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:05:54.274 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:05:54.275 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout:====================================================================================== 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout:Install 148 Packages 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 8 Packages 2024-09-12T02:05:54.276 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:05:54.277 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 226 M 2024-09-12T02:05:54.277 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-09-12T02:05:54.641 INFO:teuthology.orchestra.run.smithi026.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-12T02:05:54.678 INFO:teuthology.orchestra.run.smithi092.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T02:05:54.841 INFO:teuthology.orchestra.run.smithi026.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.4 MB/s | 897 kB 00:00 2024-09-12T02:05:54.933 INFO:teuthology.orchestra.run.smithi026.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-12T02:05:55.154 INFO:teuthology.orchestra.run.smithi092.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.8 MB/s | 897 kB 00:00 2024-09-12T02:05:55.191 INFO:teuthology.orchestra.run.smithi026.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 8.0 MB/s | 5.4 MB 00:00 2024-09-12T02:05:55.197 INFO:teuthology.orchestra.run.smithi003.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-09-12T02:05:55.204 INFO:teuthology.orchestra.run.smithi092.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.8 MB/s | 145 kB 00:00 2024-09-12T02:05:55.279 INFO:teuthology.orchestra.run.smithi092.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.4 MB/s | 5.4 MB 00:00 2024-09-12T02:05:55.325 INFO:teuthology.orchestra.run.smithi026.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-12T02:05:55.375 INFO:teuthology.orchestra.run.smithi026.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 2.4 MB 00:00 2024-09-12T02:05:55.488 INFO:teuthology.orchestra.run.smithi092.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-12T02:05:55.784 INFO:teuthology.orchestra.run.smithi026.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-12T02:05:55.905 INFO:teuthology.orchestra.run.smithi003.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 897 kB 00:00 2024-09-12T02:05:55.955 INFO:teuthology.orchestra.run.smithi092.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.2 MB/s | 2.4 MB 00:00 2024-09-12T02:05:55.989 INFO:teuthology.orchestra.run.smithi003.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-12T02:05:56.046 INFO:teuthology.orchestra.run.smithi092.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 8.4 MB/s | 4.7 MB 00:00 2024-09-12T02:05:56.067 INFO:teuthology.orchestra.run.smithi026.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-12T02:05:56.134 INFO:teuthology.orchestra.run.smithi026.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-12T02:05:56.735 INFO:teuthology.orchestra.run.smithi026.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 11 MB/s | 11 MB 00:00 2024-09-12T02:05:56.747 INFO:teuthology.orchestra.run.smithi003.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-09-12T02:05:56.810 INFO:teuthology.orchestra.run.smithi026.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.0 MB/s | 228 kB 00:00 2024-09-12T02:05:56.876 INFO:teuthology.orchestra.run.smithi026.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 407 kB/s | 27 kB 00:00 2024-09-12T02:05:56.969 INFO:teuthology.orchestra.run.smithi026.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 344 kB/s | 32 kB 00:00 2024-09-12T02:05:57.048 INFO:teuthology.orchestra.run.smithi003.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 2.3 MB/s | 2.4 MB 00:01 2024-09-12T02:05:57.102 INFO:teuthology.orchestra.run.smithi026.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.5 MB/s | 752 kB 00:00 2024-09-12T02:05:57.173 INFO:teuthology.orchestra.run.smithi003.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.6 MB/s | 1.5 MB 00:00 2024-09-12T02:05:57.194 INFO:teuthology.orchestra.run.smithi026.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 163 kB 00:00 2024-09-12T02:05:57.277 INFO:teuthology.orchestra.run.smithi026.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 127 kB 00:00 2024-09-12T02:05:57.353 INFO:teuthology.orchestra.run.smithi026.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T02:05:57.464 INFO:teuthology.orchestra.run.smithi092.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 7.5 MB/s | 11 MB 00:01 2024-09-12T02:05:57.731 INFO:teuthology.orchestra.run.smithi092.stdout:(9/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 6.7 MB/s | 21 MB 00:03 2024-09-12T02:05:57.755 INFO:teuthology.orchestra.run.smithi092.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 86 kB/s | 25 kB 00:00 2024-09-12T02:05:57.898 INFO:teuthology.orchestra.run.smithi092.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 228 kB 00:00 2024-09-12T02:05:57.970 INFO:teuthology.orchestra.run.smithi026.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 6.5 MB/s | 17 MB 00:02 2024-09-12T02:05:57.973 INFO:teuthology.orchestra.run.smithi092.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 360 kB/s | 27 kB 00:00 2024-09-12T02:05:58.045 INFO:teuthology.orchestra.run.smithi026.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 7.7 MB/s | 5.3 MB 00:00 2024-09-12T02:05:58.048 INFO:teuthology.orchestra.run.smithi092.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 423 kB/s | 32 kB 00:00 2024-09-12T02:05:58.070 INFO:teuthology.orchestra.run.smithi026.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 451 kB/s | 45 kB 00:00 2024-09-12T02:05:58.129 INFO:teuthology.orchestra.run.smithi026.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T02:05:58.163 INFO:teuthology.orchestra.run.smithi026.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.7 MB/s | 162 kB 00:00 2024-09-12T02:05:58.213 INFO:teuthology.orchestra.run.smithi026.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.7 MB/s | 321 kB 00:00 2024-09-12T02:05:58.215 INFO:teuthology.orchestra.run.smithi092.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.4 MB/s | 752 kB 00:00 2024-09-12T02:05:58.238 INFO:teuthology.orchestra.run.smithi026.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-12T02:05:58.263 INFO:teuthology.orchestra.run.smithi026.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T02:05:58.273 INFO:teuthology.orchestra.run.smithi092.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-12T02:05:58.305 INFO:teuthology.orchestra.run.smithi026.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-12T02:05:58.349 INFO:teuthology.orchestra.run.smithi092.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T02:05:58.356 INFO:teuthology.orchestra.run.smithi003.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 6.1 MB/s | 21 MB 00:03 2024-09-12T02:05:58.380 INFO:teuthology.orchestra.run.smithi026.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-12T02:05:58.489 INFO:teuthology.orchestra.run.smithi026.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 294 kB/s | 32 kB 00:00 2024-09-12T02:05:58.507 INFO:teuthology.orchestra.run.smithi092.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 3.1 MB/s | 506 kB 00:00 2024-09-12T02:05:58.598 INFO:teuthology.orchestra.run.smithi026.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 9.3 MB/s | 3.1 MB 00:00 2024-09-12T02:05:58.623 INFO:teuthology.orchestra.run.smithi026.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.1 MB/s | 146 kB 00:00 2024-09-12T02:05:58.973 INFO:teuthology.orchestra.run.smithi003.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 2.4 MB/s | 4.7 MB 00:01 2024-09-12T02:05:59.065 INFO:teuthology.orchestra.run.smithi003.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 275 kB/s | 25 kB 00:00 2024-09-12T02:05:59.123 INFO:teuthology.orchestra.run.smithi026.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.8 MB/s | 3.6 MB 00:00 2024-09-12T02:05:59.173 INFO:teuthology.orchestra.run.smithi026.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:05:59.182 INFO:teuthology.orchestra.run.smithi003.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-12T02:05:59.223 INFO:teuthology.orchestra.run.smithi026.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 990 kB/s | 49 kB 00:00 2024-09-12T02:05:59.241 INFO:teuthology.orchestra.run.smithi003.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T02:05:59.274 INFO:teuthology.orchestra.run.smithi092.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.1 MB/s | 17 MB 00:03 2024-09-12T02:05:59.282 INFO:teuthology.orchestra.run.smithi026.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 281 kB/s | 16 kB 00:00 2024-09-12T02:05:59.291 INFO:teuthology.orchestra.run.smithi003.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 542 kB/s | 27 kB 00:00 2024-09-12T02:05:59.341 INFO:teuthology.orchestra.run.smithi026.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-12T02:05:59.349 INFO:teuthology.orchestra.run.smithi003.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 545 kB/s | 32 kB 00:00 2024-09-12T02:05:59.358 INFO:teuthology.orchestra.run.smithi092.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 6.3 MB/s | 5.3 MB 00:00 2024-09-12T02:05:59.383 INFO:teuthology.orchestra.run.smithi092.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 416 kB/s | 45 kB 00:00 2024-09-12T02:05:59.425 INFO:teuthology.orchestra.run.smithi092.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T02:05:59.450 INFO:teuthology.orchestra.run.smithi026.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-12T02:05:59.454 INFO:teuthology.orchestra.run.smithi003.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 7.0 MB/s | 752 kB 00:00 2024-09-12T02:05:59.459 INFO:teuthology.orchestra.run.smithi092.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-12T02:05:59.492 INFO:teuthology.orchestra.run.smithi092.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-12T02:05:59.509 INFO:teuthology.orchestra.run.smithi003.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.0 MB/s | 163 kB 00:00 2024-09-12T02:05:59.526 INFO:teuthology.orchestra.run.smithi092.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T02:05:59.551 INFO:teuthology.orchestra.run.smithi092.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-12T02:05:59.559 INFO:teuthology.orchestra.run.smithi003.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-12T02:05:59.576 INFO:teuthology.orchestra.run.smithi092.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:05:59.617 INFO:teuthology.orchestra.run.smithi003.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-12T02:05:59.635 INFO:teuthology.orchestra.run.smithi092.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T02:05:59.702 INFO:teuthology.orchestra.run.smithi092.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 479 kB/s | 32 kB 00:00 2024-09-12T02:05:59.777 INFO:teuthology.orchestra.run.smithi092.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T02:05:59.869 INFO:teuthology.orchestra.run.smithi092.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.1 MB 00:00 2024-09-12T02:05:59.875 INFO:teuthology.orchestra.run.smithi026.stdout:(37/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 13 MB/s | 49 MB 00:03 2024-09-12T02:05:59.901 INFO:teuthology.orchestra.run.smithi026.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 40 kB/s | 18 kB 00:00 2024-09-12T02:06:00.034 INFO:teuthology.orchestra.run.smithi003.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-12T02:06:00.059 INFO:teuthology.orchestra.run.smithi026.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 5.2 MB/s | 7.4 MB 00:01 2024-09-12T02:06:00.084 INFO:teuthology.orchestra.run.smithi003.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 907 kB/s | 45 kB 00:00 2024-09-12T02:06:00.135 INFO:teuthology.orchestra.run.smithi003.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.6 MB/s | 136 kB 00:00 2024-09-12T02:06:00.144 INFO:teuthology.orchestra.run.smithi092.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 9.7 MB/s | 3.6 MB 00:00 2024-09-12T02:06:00.185 INFO:teuthology.orchestra.run.smithi003.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T02:06:00.195 INFO:teuthology.orchestra.run.smithi092.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:06:00.235 INFO:teuthology.orchestra.run.smithi003.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T02:06:00.245 INFO:teuthology.orchestra.run.smithi092.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 987 kB/s | 49 kB 00:00 2024-09-12T02:06:00.286 INFO:teuthology.orchestra.run.smithi003.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T02:06:00.312 INFO:teuthology.orchestra.run.smithi092.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 246 kB/s | 16 kB 00:00 2024-09-12T02:06:00.336 INFO:teuthology.orchestra.run.smithi003.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T02:06:00.386 INFO:teuthology.orchestra.run.smithi003.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:06:00.396 INFO:teuthology.orchestra.run.smithi092.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-12T02:06:00.421 INFO:teuthology.orchestra.run.smithi092.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.6 MB/s | 285 kB 00:00 2024-09-12T02:06:00.471 INFO:teuthology.orchestra.run.smithi092.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 365 kB/s | 18 kB 00:00 2024-09-12T02:06:00.496 INFO:teuthology.orchestra.run.smithi092.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-12T02:06:00.553 INFO:teuthology.orchestra.run.smithi003.stdout:(26/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.0 MB/s | 17 MB 00:03 2024-09-12T02:06:00.580 INFO:teuthology.orchestra.run.smithi092.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-12T02:06:00.603 INFO:teuthology.orchestra.run.smithi003.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-12T02:06:00.645 INFO:teuthology.orchestra.run.smithi003.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 768 kB/s | 32 kB 00:00 2024-09-12T02:06:00.647 INFO:teuthology.orchestra.run.smithi092.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-12T02:06:00.672 INFO:teuthology.orchestra.run.smithi092.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-12T02:06:00.675 INFO:teuthology.orchestra.run.smithi026.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 100 kB/s | 80 kB 00:00 2024-09-12T02:06:00.695 INFO:teuthology.orchestra.run.smithi003.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T02:06:00.706 INFO:teuthology.orchestra.run.smithi092.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-12T02:06:00.748 INFO:teuthology.orchestra.run.smithi092.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-12T02:06:00.790 INFO:teuthology.orchestra.run.smithi092.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-12T02:06:00.832 INFO:teuthology.orchestra.run.smithi092.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-12T02:06:00.842 INFO:teuthology.orchestra.run.smithi026.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 244 kB/s | 191 kB 00:00 2024-09-12T02:06:00.854 INFO:teuthology.orchestra.run.smithi003.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 6.6 MB/s | 3.1 MB 00:00 2024-09-12T02:06:00.874 INFO:teuthology.orchestra.run.smithi092.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-09-12T02:06:00.908 INFO:teuthology.orchestra.run.smithi092.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-12T02:06:00.913 INFO:teuthology.orchestra.run.smithi003.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 16 MB/s | 3.6 MB 00:00 2024-09-12T02:06:00.963 INFO:teuthology.orchestra.run.smithi003.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:06:00.997 INFO:teuthology.orchestra.run.smithi003.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 1.5 MB/s | 49 kB 00:00 2024-09-12T02:06:01.030 INFO:teuthology.orchestra.run.smithi003.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 492 kB/s | 16 kB 00:00 2024-09-12T02:06:01.081 INFO:teuthology.orchestra.run.smithi003.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-12T02:06:01.139 INFO:teuthology.orchestra.run.smithi003.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 13 MB/s | 765 kB 00:00 2024-09-12T02:06:01.175 INFO:teuthology.orchestra.run.smithi092.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-12T02:06:01.181 INFO:teuthology.orchestra.run.smithi003.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 439 kB/s | 18 kB 00:00 2024-09-12T02:06:01.225 INFO:teuthology.orchestra.run.smithi092.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-12T02:06:01.250 INFO:teuthology.orchestra.run.smithi092.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-12T02:06:01.275 INFO:teuthology.orchestra.run.smithi092.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-12T02:06:01.309 INFO:teuthology.orchestra.run.smithi092.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-12T02:06:01.315 INFO:teuthology.orchestra.run.smithi003.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 599 kB/s | 80 kB 00:00 2024-09-12T02:06:01.334 INFO:teuthology.orchestra.run.smithi092.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-12T02:06:01.385 INFO:teuthology.orchestra.run.smithi092.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T02:06:01.435 INFO:teuthology.orchestra.run.smithi092.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T02:06:01.476 INFO:teuthology.orchestra.run.smithi026.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 508 kB/s | 801 kB 00:01 2024-09-12T02:06:01.493 INFO:teuthology.orchestra.run.smithi092.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T02:06:01.532 INFO:teuthology.orchestra.run.smithi003.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-12T02:06:01.535 INFO:teuthology.orchestra.run.smithi026.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 295 kB/s | 253 kB 00:00 2024-09-12T02:06:01.543 INFO:teuthology.orchestra.run.smithi092.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-12T02:06:01.615 INFO:teuthology.orchestra.run.smithi003.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T02:06:01.619 INFO:teuthology.orchestra.run.smithi092.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-12T02:06:01.691 INFO:teuthology.orchestra.run.smithi003.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-12T02:06:01.791 INFO:teuthology.orchestra.run.smithi003.stdout:(42/156): ceph-mgr-diskprediction-local-19.1.1- 7.9 MB/s | 7.4 MB 00:00 2024-09-12T02:06:01.833 INFO:teuthology.orchestra.run.smithi003.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-09-12T02:06:01.917 INFO:teuthology.orchestra.run.smithi003.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T02:06:01.936 INFO:teuthology.orchestra.run.smithi092.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 1.8 MB/s | 581 kB 00:00 2024-09-12T02:06:01.975 INFO:teuthology.orchestra.run.smithi003.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 580 kB/s | 106 kB 00:00 2024-09-12T02:06:01.986 INFO:teuthology.orchestra.run.smithi092.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 492 kB/s | 24 kB 00:00 2024-09-12T02:06:02.017 INFO:teuthology.orchestra.run.smithi003.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-12T02:06:02.037 INFO:teuthology.orchestra.run.smithi092.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-12T02:06:02.076 INFO:teuthology.orchestra.run.smithi003.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-09-12T02:06:02.202 INFO:teuthology.orchestra.run.smithi026.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 147 kB/s | 106 kB 00:00 2024-09-12T02:06:02.212 INFO:teuthology.orchestra.run.smithi092.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-12T02:06:02.277 INFO:teuthology.orchestra.run.smithi026.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 182 kB/s | 135 kB 00:00 2024-09-12T02:06:02.309 INFO:teuthology.orchestra.run.smithi003.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-12T02:06:02.329 INFO:teuthology.orchestra.run.smithi092.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T02:06:02.359 INFO:teuthology.orchestra.run.smithi003.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-12T02:06:02.387 INFO:teuthology.orchestra.run.smithi092.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 661 kB/s | 38 kB 00:00 2024-09-12T02:06:02.393 INFO:teuthology.orchestra.run.smithi003.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-12T02:06:02.418 INFO:teuthology.orchestra.run.smithi003.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 370 kB/s | 22 kB 00:00 2024-09-12T02:06:02.437 INFO:teuthology.orchestra.run.smithi092.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-12T02:06:02.452 INFO:teuthology.orchestra.run.smithi003.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-12T02:06:02.477 INFO:teuthology.orchestra.run.smithi003.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-12T02:06:02.544 INFO:teuthology.orchestra.run.smithi003.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-09-12T02:06:02.595 INFO:teuthology.orchestra.run.smithi003.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-12T02:06:02.596 INFO:teuthology.orchestra.run.smithi092.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-12T02:06:02.653 INFO:teuthology.orchestra.run.smithi003.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T02:06:02.680 INFO:teuthology.orchestra.run.smithi092.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T02:06:02.703 INFO:teuthology.orchestra.run.smithi003.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-12T02:06:02.755 INFO:teuthology.orchestra.run.smithi092.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-12T02:06:02.804 INFO:teuthology.orchestra.run.smithi003.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-12T02:06:02.862 INFO:teuthology.orchestra.run.smithi003.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-12T02:06:02.894 INFO:teuthology.orchestra.run.smithi026.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 612 kB/s | 1.2 MB 00:02 2024-09-12T02:06:02.904 INFO:teuthology.orchestra.run.smithi003.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-12T02:06:02.919 INFO:teuthology.orchestra.run.smithi026.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 174 kB/s | 125 kB 00:00 2024-09-12T02:06:02.955 INFO:teuthology.orchestra.run.smithi003.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-12T02:06:03.086 INFO:teuthology.orchestra.run.smithi026.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 266 kB/s | 215 kB 00:00 2024-09-12T02:06:03.138 INFO:teuthology.orchestra.run.smithi003.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-12T02:06:03.189 INFO:teuthology.orchestra.run.smithi092.stdout:(69/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 8.9 MB/s | 49 MB 00:05 2024-09-12T02:06:03.214 INFO:teuthology.orchestra.run.smithi092.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 299 kB/s | 137 kB 00:00 2024-09-12T02:06:03.255 INFO:teuthology.orchestra.run.smithi003.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T02:06:03.289 INFO:teuthology.orchestra.run.smithi092.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T02:06:03.340 INFO:teuthology.orchestra.run.smithi092.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-12T02:06:03.347 INFO:teuthology.orchestra.run.smithi003.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 420 kB/s | 38 kB 00:00 2024-09-12T02:06:03.515 INFO:teuthology.orchestra.run.smithi092.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-12T02:06:03.574 INFO:teuthology.orchestra.run.smithi092.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T02:06:03.632 INFO:teuthology.orchestra.run.smithi092.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T02:06:03.691 INFO:teuthology.orchestra.run.smithi092.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-12T02:06:03.749 INFO:teuthology.orchestra.run.smithi092.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-12T02:06:03.778 INFO:teuthology.orchestra.run.smithi026.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 155 kB/s | 107 kB 00:00 2024-09-12T02:06:03.806 INFO:teuthology.orchestra.run.smithi003.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-09-12T02:06:03.816 INFO:teuthology.orchestra.run.smithi092.stdout:(78/156): postgresql-server-13.16-1.el9.x86_64. 9.2 MB/s | 5.8 MB 00:00 2024-09-12T02:06:03.842 INFO:teuthology.orchestra.run.smithi092.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-09-12T02:06:03.875 INFO:teuthology.orchestra.run.smithi092.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T02:06:03.909 INFO:teuthology.orchestra.run.smithi092.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-09-12T02:06:04.017 INFO:teuthology.orchestra.run.smithi092.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-12T02:06:04.084 INFO:teuthology.orchestra.run.smithi092.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-12T02:06:04.103 INFO:teuthology.orchestra.run.smithi026.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 570 kB/s | 689 kB 00:01 2024-09-12T02:06:04.110 INFO:teuthology.orchestra.run.smithi092.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T02:06:04.135 INFO:teuthology.orchestra.run.smithi092.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T02:06:04.139 INFO:teuthology.orchestra.run.smithi003.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-12T02:06:04.152 INFO:teuthology.orchestra.run.smithi092.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T02:06:04.162 INFO:teuthology.orchestra.run.smithi026.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 759 kB/s | 943 kB 00:01 2024-09-12T02:06:04.215 INFO:teuthology.orchestra.run.smithi003.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T02:06:04.218 INFO:teuthology.orchestra.run.smithi092.stdout:(87/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.7 MB/s | 4.9 MB 00:01 2024-09-12T02:06:04.244 INFO:teuthology.orchestra.run.smithi092.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-12T02:06:04.269 INFO:teuthology.orchestra.run.smithi092.stdout:(89/156): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-12T02:06:04.306 INFO:teuthology.orchestra.run.smithi003.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.2 MB/s | 4.9 MB 00:00 2024-09-12T02:06:04.311 INFO:teuthology.orchestra.run.smithi092.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.9 MB/s | 279 kB 00:00 2024-09-12T02:06:04.328 INFO:teuthology.orchestra.run.smithi092.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 607 kB/s | 51 kB 00:00 2024-09-12T02:06:04.329 INFO:teuthology.orchestra.run.smithi026.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-12T02:06:04.340 INFO:teuthology.orchestra.run.smithi003.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-12T02:06:04.345 INFO:teuthology.orchestra.run.smithi092.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-12T02:06:04.365 INFO:teuthology.orchestra.run.smithi003.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T02:06:04.429 INFO:teuthology.orchestra.run.smithi092.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 501 kB/s | 42 kB 00:00 2024-09-12T02:06:04.446 INFO:teuthology.orchestra.run.smithi092.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-09-12T02:06:04.496 INFO:teuthology.orchestra.run.smithi092.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-12T02:06:04.513 INFO:teuthology.orchestra.run.smithi092.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-09-12T02:06:04.538 INFO:teuthology.orchestra.run.smithi092.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-12T02:06:04.541 INFO:teuthology.orchestra.run.smithi003.stdout:(71/156): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-09-12T02:06:04.580 INFO:teuthology.orchestra.run.smithi092.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-12T02:06:04.591 INFO:teuthology.orchestra.run.smithi003.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-12T02:06:04.597 INFO:teuthology.orchestra.run.smithi092.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-09-12T02:06:04.631 INFO:teuthology.orchestra.run.smithi092.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T02:06:04.681 INFO:teuthology.orchestra.run.smithi092.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 758 kB/s | 64 kB 00:00 2024-09-12T02:06:04.712 INFO:teuthology.orchestra.run.smithi026.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-12T02:06:04.766 INFO:teuthology.orchestra.run.smithi003.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-09-12T02:06:04.816 INFO:teuthology.orchestra.run.smithi003.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T02:06:04.840 INFO:teuthology.orchestra.run.smithi092.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-12T02:06:04.865 INFO:teuthology.orchestra.run.smithi092.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 191 kB/s | 45 kB 00:00 2024-09-12T02:06:04.875 INFO:teuthology.orchestra.run.smithi003.stdout:(75/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-09-12T02:06:04.901 INFO:teuthology.orchestra.run.smithi003.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.8 MB/s | 245 kB 00:00 2024-09-12T02:06:04.915 INFO:teuthology.orchestra.run.smithi092.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:06:04.934 INFO:teuthology.orchestra.run.smithi003.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-12T02:06:04.940 INFO:teuthology.orchestra.run.smithi092.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-09-12T02:06:04.959 INFO:teuthology.orchestra.run.smithi003.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-12T02:06:04.974 INFO:teuthology.orchestra.run.smithi092.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.1 MB/s | 309 kB 00:00 2024-09-12T02:06:04.984 INFO:teuthology.orchestra.run.smithi003.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-12T02:06:04.999 INFO:teuthology.orchestra.run.smithi092.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-12T02:06:05.010 INFO:teuthology.orchestra.run.smithi003.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T02:06:05.025 INFO:teuthology.orchestra.run.smithi092.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:06:05.035 INFO:teuthology.orchestra.run.smithi003.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-09-12T02:06:05.050 INFO:teuthology.orchestra.run.smithi092.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:06:05.063 INFO:teuthology.orchestra.run.smithi026.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 327 kB/s | 313 kB 00:00 2024-09-12T02:06:05.083 INFO:teuthology.orchestra.run.smithi092.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-12T02:06:05.102 INFO:teuthology.orchestra.run.smithi003.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-12T02:06:05.117 INFO:teuthology.orchestra.run.smithi092.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-12T02:06:05.150 INFO:teuthology.orchestra.run.smithi092.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-12T02:06:05.169 INFO:teuthology.orchestra.run.smithi003.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-12T02:06:05.188 INFO:teuthology.orchestra.run.smithi026.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-12T02:06:05.194 INFO:teuthology.orchestra.run.smithi003.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T02:06:05.201 INFO:teuthology.orchestra.run.smithi092.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 10 MB/s | 839 kB 00:00 2024-09-12T02:06:05.219 INFO:teuthology.orchestra.run.smithi003.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T02:06:05.226 INFO:teuthology.orchestra.run.smithi092.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:06:05.245 INFO:teuthology.orchestra.run.smithi003.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-12T02:06:05.260 INFO:teuthology.orchestra.run.smithi092.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-12T02:06:05.278 INFO:teuthology.orchestra.run.smithi003.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T02:06:05.285 INFO:teuthology.orchestra.run.smithi092.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-09-12T02:06:05.304 INFO:teuthology.orchestra.run.smithi003.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T02:06:05.319 INFO:teuthology.orchestra.run.smithi092.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-12T02:06:05.411 INFO:teuthology.orchestra.run.smithi092.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-12T02:06:05.436 INFO:teuthology.orchestra.run.smithi092.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-09-12T02:06:05.478 INFO:teuthology.orchestra.run.smithi092.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-09-12T02:06:05.503 INFO:teuthology.orchestra.run.smithi092.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-12T02:06:05.528 INFO:teuthology.orchestra.run.smithi092.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-12T02:06:05.530 INFO:teuthology.orchestra.run.smithi026.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-12T02:06:05.545 INFO:teuthology.orchestra.run.smithi092.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 646 kB/s | 11 kB 00:00 2024-09-12T02:06:05.571 INFO:teuthology.orchestra.run.smithi092.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-12T02:06:05.587 INFO:teuthology.orchestra.run.smithi092.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-09-12T02:06:05.604 INFO:teuthology.orchestra.run.smithi092.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 696 kB/s | 23 kB 00:00 2024-09-12T02:06:05.621 INFO:teuthology.orchestra.run.smithi092.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-12T02:06:05.646 INFO:teuthology.orchestra.run.smithi092.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-09-12T02:06:05.672 INFO:teuthology.orchestra.run.smithi092.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:06:05.697 INFO:teuthology.orchestra.run.smithi092.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:06:05.723 INFO:teuthology.orchestra.run.smithi092.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-12T02:06:05.729 INFO:teuthology.orchestra.run.smithi003.stdout:(89/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 7.3 MB/s | 49 MB 00:06 2024-09-12T02:06:05.754 INFO:teuthology.orchestra.run.smithi003.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 113 kB/s | 51 kB 00:00 2024-09-12T02:06:05.765 INFO:teuthology.orchestra.run.smithi092.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-12T02:06:05.779 INFO:teuthology.orchestra.run.smithi003.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 557 kB/s | 279 kB 00:00 2024-09-12T02:06:05.790 INFO:teuthology.orchestra.run.smithi092.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-12T02:06:05.815 INFO:teuthology.orchestra.run.smithi092.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-12T02:06:05.830 INFO:teuthology.orchestra.run.smithi003.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-12T02:06:05.840 INFO:teuthology.orchestra.run.smithi092.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T02:06:05.866 INFO:teuthology.orchestra.run.smithi092.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T02:06:05.888 INFO:teuthology.orchestra.run.smithi003.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-12T02:06:05.891 INFO:teuthology.orchestra.run.smithi092.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-12T02:06:05.913 INFO:teuthology.orchestra.run.smithi003.stdout:(94/156): python3-requests-oauthlib-1.3.0-12.el 291 kB/s | 54 kB 00:00 2024-09-12T02:06:05.916 INFO:teuthology.orchestra.run.smithi092.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:06:05.930 INFO:teuthology.orchestra.run.smithi026.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-09-12T02:06:05.941 INFO:teuthology.orchestra.run.smithi092.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:06:05.947 INFO:teuthology.orchestra.run.smithi003.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:06:05.964 INFO:teuthology.orchestra.run.smithi003.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-12T02:06:05.967 INFO:teuthology.orchestra.run.smithi092.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-12T02:06:05.992 INFO:teuthology.orchestra.run.smithi092.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T02:06:05.997 INFO:teuthology.orchestra.run.smithi003.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-12T02:06:06.017 INFO:teuthology.orchestra.run.smithi092.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:06:06.047 INFO:teuthology.orchestra.run.smithi003.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T02:06:06.067 INFO:teuthology.orchestra.run.smithi092.stdout:(143/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-09-12T02:06:06.081 INFO:teuthology.orchestra.run.smithi003.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-12T02:06:06.093 INFO:teuthology.orchestra.run.smithi092.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-12T02:06:06.106 INFO:teuthology.orchestra.run.smithi003.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T02:06:06.118 INFO:teuthology.orchestra.run.smithi092.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-12T02:06:06.132 INFO:teuthology.orchestra.run.smithi003.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T02:06:06.143 INFO:teuthology.orchestra.run.smithi092.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:06:06.163 INFO:teuthology.orchestra.run.smithi026.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-12T02:06:06.169 INFO:teuthology.orchestra.run.smithi092.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-12T02:06:06.219 INFO:teuthology.orchestra.run.smithi092.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:06:06.291 INFO:teuthology.orchestra.run.smithi003.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-12T02:06:06.316 INFO:teuthology.orchestra.run.smithi003.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 214 kB/s | 45 kB 00:00 2024-09-12T02:06:06.366 INFO:teuthology.orchestra.run.smithi003.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:06:06.391 INFO:teuthology.orchestra.run.smithi003.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-12T02:06:06.411 INFO:teuthology.orchestra.run.smithi092.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-12T02:06:06.417 INFO:teuthology.orchestra.run.smithi003.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-12T02:06:06.470 INFO:teuthology.orchestra.run.smithi092.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T02:06:06.520 INFO:teuthology.orchestra.run.smithi092.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-09-12T02:06:06.525 INFO:teuthology.orchestra.run.smithi003.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T02:06:06.551 INFO:teuthology.orchestra.run.smithi003.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 53 kB/s | 22 kB 00:00 2024-09-12T02:06:06.562 INFO:teuthology.orchestra.run.smithi092.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-12T02:06:06.576 INFO:teuthology.orchestra.run.smithi003.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-12T02:06:06.587 INFO:teuthology.orchestra.run.smithi092.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 348 kB/s | 41 kB 00:00 2024-09-12T02:06:06.609 INFO:teuthology.orchestra.run.smithi003.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T02:06:06.635 INFO:teuthology.orchestra.run.smithi003.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-12T02:06:06.637 INFO:teuthology.orchestra.run.smithi092.stdout:(154/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.7 MB/s | 3.2 MB 00:00 2024-09-12T02:06:06.660 INFO:teuthology.orchestra.run.smithi003.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-09-12T02:06:06.680 INFO:teuthology.orchestra.run.smithi092.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-12T02:06:06.703 INFO:teuthology.orchestra.run.smithi003.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-12T02:06:06.727 INFO:teuthology.orchestra.run.smithi003.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:06:06.752 INFO:teuthology.orchestra.run.smithi003.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:06:06.778 INFO:teuthology.orchestra.run.smithi003.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-12T02:06:06.788 INFO:teuthology.orchestra.run.smithi092.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-12T02:06:06.794 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:06:06.794 INFO:teuthology.orchestra.run.smithi092.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-12T02:06:06.803 INFO:teuthology.orchestra.run.smithi003.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-12T02:06:06.828 INFO:teuthology.orchestra.run.smithi003.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-12T02:06:06.862 INFO:teuthology.orchestra.run.smithi003.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T02:06:06.887 INFO:teuthology.orchestra.run.smithi003.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-12T02:06:06.913 INFO:teuthology.orchestra.run.smithi003.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T02:06:06.938 INFO:teuthology.orchestra.run.smithi003.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-12T02:06:06.963 INFO:teuthology.orchestra.run.smithi003.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-12T02:06:06.972 INFO:teuthology.orchestra.run.smithi026.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 636 kB/s | 662 kB 00:01 2024-09-12T02:06:06.988 INFO:teuthology.orchestra.run.smithi003.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-12T02:06:07.014 INFO:teuthology.orchestra.run.smithi003.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-12T02:06:07.039 INFO:teuthology.orchestra.run.smithi003.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-12T02:06:07.039 INFO:teuthology.orchestra.run.smithi026.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 670 kB/s | 581 kB 00:00 2024-09-12T02:06:07.064 INFO:teuthology.orchestra.run.smithi003.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-12T02:06:07.106 INFO:teuthology.orchestra.run.smithi003.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 24 MB/s | 1.0 MB 00:00 2024-09-12T02:06:07.123 INFO:teuthology.orchestra.run.smithi026.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.1 MB/s | 3.0 MB 00:02 2024-09-12T02:06:07.131 INFO:teuthology.orchestra.run.smithi003.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:06:07.156 INFO:teuthology.orchestra.run.smithi003.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:06:07.198 INFO:teuthology.orchestra.run.smithi003.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-09-12T02:06:07.241 INFO:teuthology.orchestra.run.smithi003.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-09-12T02:06:07.266 INFO:teuthology.orchestra.run.smithi003.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-12T02:06:07.291 INFO:teuthology.orchestra.run.smithi003.stdout:(134/156): protobuf-compiler-3.14.0-13.el9.x86_ 864 kB/s | 863 kB 00:00 2024-09-12T02:06:07.324 INFO:teuthology.orchestra.run.smithi003.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-12T02:06:07.350 INFO:teuthology.orchestra.run.smithi003.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-12T02:06:07.375 INFO:teuthology.orchestra.run.smithi003.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-09-12T02:06:07.400 INFO:teuthology.orchestra.run.smithi003.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 419 kB/s | 31 kB 00:00 2024-09-12T02:06:07.426 INFO:teuthology.orchestra.run.smithi003.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T02:06:07.451 INFO:teuthology.orchestra.run.smithi003.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-12T02:06:07.476 INFO:teuthology.orchestra.run.smithi003.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-12T02:06:07.501 INFO:teuthology.orchestra.run.smithi003.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 989 kB/s | 74 kB 00:00 2024-09-12T02:06:07.506 INFO:teuthology.orchestra.run.smithi026.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-12T02:06:07.526 INFO:teuthology.orchestra.run.smithi003.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T02:06:07.552 INFO:teuthology.orchestra.run.smithi003.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T02:06:07.569 INFO:teuthology.orchestra.run.smithi003.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-09-12T02:06:07.594 INFO:teuthology.orchestra.run.smithi003.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-12T02:06:07.619 INFO:teuthology.orchestra.run.smithi003.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-12T02:06:07.669 INFO:teuthology.orchestra.run.smithi003.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:06:07.723 INFO:teuthology.orchestra.run.smithi026.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-09-12T02:06:07.845 INFO:teuthology.orchestra.run.smithi003.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-12T02:06:07.937 INFO:teuthology.orchestra.run.smithi003.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-09-12T02:06:08.028 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-09-12T02:06:08.029 INFO:teuthology.orchestra.run.smithi003.stdout:(151/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.9 MB/s | 3.4 MB 00:00 2024-09-12T02:06:08.054 INFO:teuthology.orchestra.run.smithi003.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-12T02:06:08.090 INFO:teuthology.orchestra.run.smithi026.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 302 kB/s | 292 kB 00:00 2024-09-12T02:06:08.104 INFO:teuthology.orchestra.run.smithi003.stdout:(153/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.5 MB/s | 3.2 MB 00:00 2024-09-12T02:06:08.146 INFO:teuthology.orchestra.run.smithi003.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-12T02:06:08.171 INFO:teuthology.orchestra.run.smithi003.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-12T02:06:08.178 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-09-12T02:06:08.178 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-09-12T02:06:08.282 INFO:teuthology.orchestra.run.smithi026.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-12T02:06:08.355 INFO:teuthology.orchestra.run.smithi003.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-12T02:06:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:06:08.361 INFO:teuthology.orchestra.run.smithi003.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-12T02:06:08.440 INFO:teuthology.orchestra.run.smithi026.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 202 kB/s | 188 kB 00:00 2024-09-12T02:06:09.642 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-09-12T02:06:09.810 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-09-12T02:06:09.810 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-09-12T02:06:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-09-12T02:06:09.975 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-09-12T02:06:10.341 INFO:teuthology.orchestra.run.smithi026.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 855 kB/s | 1.6 MB 00:01 2024-09-12T02:06:10.549 INFO:teuthology.orchestra.run.smithi026.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.0 MB/s | 4.9 MB 00:02 2024-09-12T02:06:10.608 INFO:teuthology.orchestra.run.smithi026.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-09-12T02:06:11.266 INFO:teuthology.orchestra.run.smithi026.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-12T02:06:12.041 INFO:teuthology.orchestra.run.smithi026.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 535 kB/s | 910 kB 00:01 2024-09-12T02:06:12.106 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-09-12T02:06:12.106 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-09-12T02:06:12.192 INFO:teuthology.orchestra.run.smithi026.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 3.6 MB/s | 5.8 MB 00:01 2024-09-12T02:06:12.700 INFO:teuthology.orchestra.run.smithi026.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 721 kB/s | 1.0 MB 00:01 2024-09-12T02:06:12.725 INFO:teuthology.orchestra.run.smithi026.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 128 kB/s | 87 kB 00:00 2024-09-12T02:06:13.351 INFO:teuthology.orchestra.run.smithi026.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-12T02:06:13.376 INFO:teuthology.orchestra.run.smithi026.stdout:(76/156): python3-colorama-0.4.6-3.el9.noarch.r 91 kB/s | 61 kB 00:00 2024-09-12T02:06:13.525 INFO:teuthology.orchestra.run.smithi026.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 306 kB/s | 245 kB 00:00 2024-09-12T02:06:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-12T02:06:14.052 INFO:teuthology.orchestra.run.smithi026.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-09-12T02:06:14.235 INFO:teuthology.orchestra.run.smithi026.stdout:(80/156): python3-jinja2-2.11.3-6.el9.noarch.rp 281 kB/s | 248 kB 00:00 2024-09-12T02:06:14.652 INFO:teuthology.orchestra.run.smithi026.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 58 kB/s | 35 kB 00:00 2024-09-12T02:06:14.711 INFO:teuthology.orchestra.run.smithi026.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 217 kB/s | 172 kB 00:00 2024-09-12T02:06:15.411 INFO:teuthology.orchestra.run.smithi026.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 330 kB/s | 250 kB 00:00 2024-09-12T02:06:15.470 INFO:teuthology.orchestra.run.smithi026.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 293 kB/s | 222 kB 00:00 2024-09-12T02:06:15.620 INFO:teuthology.orchestra.run.smithi026.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-09-12T02:06:16.070 INFO:teuthology.orchestra.run.smithi026.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 70 kB/s | 42 kB 00:00 2024-09-12T02:06:16.112 INFO:teuthology.orchestra.run.smithi026.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-09-12T02:06:16.260 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-09-12T02:06:16.357 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:06:16.404 INFO:teuthology.orchestra.run.smithi026.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 344 kB/s | 269 kB 00:00 2024-09-12T02:06:16.489 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:06:16.652 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:06:16.692 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:06:16.717 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:06:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:06:16.771 INFO:teuthology.orchestra.run.smithi026.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 227 kB/s | 159 kB 00:00 2024-09-12T02:06:16.946 INFO:teuthology.orchestra.run.smithi026.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 335 kB/s | 279 kB 00:00 2024-09-12T02:06:17.004 INFO:teuthology.orchestra.run.smithi026.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-12T02:06:17.099 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:06:17.218 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:17.340 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:17.369 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:17.396 INFO:teuthology.orchestra.run.smithi026.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 86 kB/s | 54 kB 00:00 2024-09-12T02:06:17.460 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:17.490 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:06:17.605 INFO:teuthology.orchestra.run.smithi026.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 70 kB/s | 42 kB 00:00 2024-09-12T02:06:17.638 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:06:17.686 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:06:17.714 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:17.802 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:17.832 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:17.878 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:17.899 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:17.924 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:17.964 INFO:teuthology.orchestra.run.smithi026.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-09-12T02:06:18.117 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:18.262 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:18.410 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:06:18.455 INFO:teuthology.orchestra.run.smithi026.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 260 kB/s | 221 kB 00:00 2024-09-12T02:06:18.514 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:18.514 INFO:teuthology.orchestra.run.smithi026.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 65 kB/s | 36 kB 00:00 2024-09-12T02:06:18.591 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:18.698 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-09-12T02:06:18.829 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:18.841 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:06:18.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:06:18.943 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:06:18.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:06:18.975 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:06:19.000 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:06:19.031 INFO:teuthology.orchestra.run.smithi026.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-12T02:06:19.038 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:06:19.052 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:06:19.074 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:06:19.117 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:06:19.191 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:06:19.235 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:06:19.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:06:19.324 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:06:19.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:06:19.387 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:06:19.400 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:06:19.454 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:06:19.510 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:19.517 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:06:19.556 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:06:19.589 INFO:teuthology.orchestra.run.smithi026.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 283 kB/s | 304 kB 00:01 2024-09-12T02:06:19.591 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:06:19.635 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:19.648 INFO:teuthology.orchestra.run.smithi026.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-12T02:06:19.661 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:19.669 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:06:19.702 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:06:19.754 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:19.754 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:06:19.782 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:06:19.783 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:06:19.853 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:06:19.898 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:06:19.956 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:06:19.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:06:20.012 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:06:20.039 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:20.120 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:06:20.146 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:20.173 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:20.174 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:06:20.220 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:20.248 INFO:teuthology.orchestra.run.smithi026.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 145 kB/s | 96 kB 00:00 2024-09-12T02:06:20.249 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:20.273 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:20.365 INFO:teuthology.orchestra.run.smithi026.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-09-12T02:06:20.470 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:20.590 INFO:teuthology.orchestra.run.smithi026.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-09-12T02:06:20.642 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:20.704 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:06:20.724 INFO:teuthology.orchestra.run.smithi026.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-09-12T02:06:20.764 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:06:20.783 INFO:teuthology.orchestra.run.smithi026.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T02:06:20.795 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:06:20.833 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:06:20.837 INFO:teuthology.orchestra.run.smithi026.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.5 MB/s | 309 kB 00:00 2024-09-12T02:06:20.867 INFO:teuthology.orchestra.run.smithi026.stdout:(106/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-12T02:06:20.884 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:06:20.884 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:20.892 INFO:teuthology.orchestra.run.smithi026.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 359 kB/s | 19 kB 00:00 2024-09-12T02:06:20.922 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:06:20.926 INFO:teuthology.orchestra.run.smithi026.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:06:20.951 INFO:teuthology.orchestra.run.smithi026.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-12T02:06:20.966 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:20.971 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:06:20.976 INFO:teuthology.orchestra.run.smithi026.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T02:06:21.001 INFO:teuthology.orchestra.run.smithi026.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-12T02:06:21.043 INFO:teuthology.orchestra.run.smithi026.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-12T02:06:21.085 INFO:teuthology.orchestra.run.smithi026.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T02:06:21.110 INFO:teuthology.orchestra.run.smithi026.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:06:21.117 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:06:21.127 INFO:teuthology.orchestra.run.smithi026.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-12T02:06:21.144 INFO:teuthology.orchestra.run.smithi026.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 850 kB/s | 14 kB 00:00 2024-09-12T02:06:21.178 INFO:teuthology.orchestra.run.smithi026.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-12T02:06:21.183 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:06:21.203 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:21.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:06:21.245 INFO:teuthology.orchestra.run.smithi026.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-12T02:06:21.270 INFO:teuthology.orchestra.run.smithi026.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-12T02:06:21.295 INFO:teuthology.orchestra.run.smithi026.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T02:06:21.310 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:06:21.317 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:06:21.321 INFO:teuthology.orchestra.run.smithi026.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-12T02:06:21.342 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:06:21.354 INFO:teuthology.orchestra.run.smithi026.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T02:06:21.378 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:06:21.380 INFO:teuthology.orchestra.run.smithi026.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-12T02:06:21.405 INFO:teuthology.orchestra.run.smithi026.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-12T02:06:21.422 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:06:21.424 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:06:21.430 INFO:teuthology.orchestra.run.smithi026.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-12T02:06:21.455 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:06:21.455 INFO:teuthology.orchestra.run.smithi026.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 786 kB/s | 19 kB 00:00 2024-09-12T02:06:21.460 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:06:21.490 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:06:21.497 INFO:teuthology.orchestra.run.smithi026.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-09-12T02:06:21.503 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:06:21.523 INFO:teuthology.orchestra.run.smithi026.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-09-12T02:06:21.548 INFO:teuthology.orchestra.run.smithi026.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:06:21.554 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:06:21.565 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:06:21.573 INFO:teuthology.orchestra.run.smithi026.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:06:21.599 INFO:teuthology.orchestra.run.smithi026.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T02:06:21.615 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:06:21.632 INFO:teuthology.orchestra.run.smithi026.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T02:06:21.655 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:06:21.665 INFO:teuthology.orchestra.run.smithi026.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-12T02:06:21.691 INFO:teuthology.orchestra.run.smithi026.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-12T02:06:21.698 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:06:21.707 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:06:21.716 INFO:teuthology.orchestra.run.smithi026.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T02:06:21.740 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:06:21.742 INFO:teuthology.orchestra.run.smithi026.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 559 kB/s | 28 kB 00:00 2024-09-12T02:06:21.767 INFO:teuthology.orchestra.run.smithi026.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-12T02:06:21.767 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:06:21.792 INFO:teuthology.orchestra.run.smithi026.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T02:06:21.818 INFO:teuthology.orchestra.run.smithi026.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:06:21.838 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:06:21.843 INFO:teuthology.orchestra.run.smithi026.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-12T02:06:21.868 INFO:teuthology.orchestra.run.smithi026.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T02:06:21.891 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:06:21.893 INFO:teuthology.orchestra.run.smithi026.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T02:06:21.918 INFO:teuthology.orchestra.run.smithi026.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T02:06:21.920 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:06:21.944 INFO:teuthology.orchestra.run.smithi026.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 797 kB/s | 20 kB 00:00 2024-09-12T02:06:21.964 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:06:21.969 INFO:teuthology.orchestra.run.smithi026.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-12T02:06:21.994 INFO:teuthology.orchestra.run.smithi026.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:06:21.998 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:06:22.042 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:06:22.045 INFO:teuthology.orchestra.run.smithi026.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T02:06:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:06:22.074 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:06:22.110 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:06:22.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:06:22.154 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:06:22.189 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:06:22.210 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:06:22.229 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:06:22.253 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:06:22.254 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:06:22.279 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:06:22.329 INFO:teuthology.orchestra.run.smithi026.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-12T02:06:22.330 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:06:22.356 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:06:22.395 INFO:teuthology.orchestra.run.smithi026.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-09-12T02:06:22.477 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:06:22.481 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:06:22.528 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:06:22.530 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:06:22.556 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:06:22.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:06:22.637 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:06:22.677 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:06:22.707 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:06:22.750 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:06:22.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:06:22.825 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:06:22.860 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:06:22.946 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:06:22.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:06:23.030 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:06:23.087 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:06:23.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:06:23.186 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:06:23.197 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:06:23.221 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:06:23.263 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:06:23.293 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:06:23.336 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:06:23.502 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:06:23.546 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:06:23.602 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:06:23.654 INFO:teuthology.orchestra.run.smithi026.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-09-12T02:06:23.685 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:06:23.722 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:06:23.758 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:06:23.806 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:06:23.841 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:06:23.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:06:23.927 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:06:23.952 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:06:24.003 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:06:24.036 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:06:24.070 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:06:24.111 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:06:24.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:06:24.185 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:06:24.212 INFO:teuthology.orchestra.run.smithi026.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 73 kB/s | 41 kB 00:00 2024-09-12T02:06:24.238 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:06:24.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:06:24.382 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:06:24.388 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:06:24.515 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:06:24.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:06:24.567 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:06:24.593 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:06:24.597 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:06:24.625 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:06:24.650 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:06:24.674 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:06:24.750 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:06:24.881 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:06:24.916 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:06:24.944 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:06:24.980 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:06:25.020 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:06:25.061 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:06:25.089 INFO:teuthology.orchestra.run.smithi026.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 284 kB/s | 249 kB 00:00 2024-09-12T02:06:25.090 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:06:25.121 INFO:teuthology.orchestra.run.smithi026.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 678 kB/s | 1.8 MB 00:02 2024-09-12T02:06:25.125 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:06:25.158 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:06:25.200 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:06:25.234 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:06:25.316 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:06:25.324 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:06:25.373 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:06:25.404 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:06:25.437 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:06:25.512 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:06:25.564 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:06:26.326 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:06:26.377 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:06:26.411 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:06:26.513 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:06:26.566 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:06:26.611 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:06:26.651 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:06:26.719 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:06:26.759 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:06:26.786 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:06:26.819 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:06:26.975 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:06:27.128 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:06:27.174 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:27.229 INFO:teuthology.orchestra.run.smithi026.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 647 kB/s | 1.4 MB 00:02 2024-09-12T02:06:27.261 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:06:27.274 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:27.294 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:06:27.312 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:06:27.357 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:06:27.381 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:06:27.834 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:06:27.869 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:27.940 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:27.970 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:06:28.085 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:06:28.205 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:06:28.546 INFO:teuthology.orchestra.run.smithi026.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 715 kB/s | 2.4 MB 00:03 2024-09-12T02:06:29.263 INFO:teuthology.orchestra.run.smithi026.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:12 2024-09-12T02:06:29.270 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:06:29.270 INFO:teuthology.orchestra.run.smithi026.stdout:Total 6.4 MB/s | 226 MB 00:35 2024-09-12T02:06:29.553 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:06:29.639 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:06:29.668 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:06:29.701 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:06:30.026 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:06:30.072 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:30.172 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:30.202 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:06:30.233 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:30.248 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:06:30.275 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:30.511 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-09-12T02:06:30.680 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-09-12T02:06:30.680 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-09-12T02:06:30.750 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:06:30.783 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:30.821 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:30.854 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:30.861 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:30.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:06:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:30.954 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:06:30.954 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:06:30.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:30.987 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:06:32.592 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-09-12T02:06:32.593 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-09-12T02:06:33.338 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:33.372 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:33.953 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:33.992 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:34.092 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:34.092 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:06:34.092 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:06:34.092 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:34.135 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:06:39.056 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-09-12T02:06:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:06:39.231 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:06:39.284 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:06:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:06:39.343 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:06:39.386 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:06:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:06:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:39.968 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:06:40.003 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:06:40.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:06:40.282 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:06:40.338 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:06:40.365 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:40.471 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:06:40.499 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:40.562 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:06:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:40.614 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:06:40.810 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:40.973 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:06:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:06:41.210 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:41.496 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:06:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:06:41.706 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:06:41.747 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:06:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:06:41.857 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:06:41.908 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:06:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:06:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:06:42.024 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:06:42.052 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:06:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:06:42.175 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:06:42.204 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:06:42.240 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:06:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:06:42.354 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:06:42.398 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:06:42.425 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:06:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:06:42.549 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:06:42.626 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-09-12T02:06:42.713 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-09-12T02:06:42.714 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-09-12T02:06:42.714 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-09-12T02:06:42.714 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:42.780 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:06:42.836 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:06:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:06:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:06:42.896 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:42.896 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:06:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:43.396 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:06:43.482 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:06:43.517 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:06:43.560 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:06:43.598 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:06:43.640 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:06:43.817 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:06:43.876 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:06:43.933 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:06:44.006 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:06:44.044 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:06:44.080 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:06:44.127 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:06:44.162 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:06:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:06:44.248 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:06:44.374 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:06:44.390 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:06:44.401 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:06:44.401 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:44.402 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:06:44.402 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:06:44.402 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:06:44.402 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:44.441 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:06:44.483 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:06:44.589 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:06:44.624 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:06:44.634 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:06:44.658 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:06:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:06:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:06:44.658 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:06:44.659 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:44.664 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:06:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:06:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:06:44.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:06:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:06:44.963 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:06:44.991 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:06:45.025 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:06:45.065 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:06:45.093 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:06:45.104 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:06:45.117 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:06:45.117 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:45.117 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:06:45.117 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:06:45.117 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:06:45.118 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:45.134 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:06:45.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:06:45.202 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:06:45.224 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:06:45.244 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:06:45.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:06:45.286 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:06:45.344 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:06:45.376 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:06:45.434 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:06:45.464 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:06:45.538 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:06:45.571 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:06:45.594 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:06:45.633 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:06:45.672 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:06:45.761 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:06:45.901 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:06:45.944 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:06:45.976 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:06:46.004 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:06:46.398 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:06:46.445 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:06:46.446 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:06:46.479 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:06:46.488 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:06:46.518 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:06:46.579 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:06:46.583 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:06:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-09-12T02:06:46.621 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-09-12T02:06:46.622 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-09-12T02:06:46.622 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-09-12T02:06:46.622 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:46.635 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:06:46.680 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:06:46.780 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:06:46.803 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:06:46.821 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:06:46.821 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:46.821 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:06:46.821 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:46.823 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:06:47.014 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:06:47.045 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:06:47.125 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:06:47.651 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:06:47.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:06:47.825 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:06:47.948 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:06:48.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:06:48.393 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:06:48.394 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:48.394 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:06:48.394 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:06:48.394 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:06:48.394 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:48.632 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:06:48.657 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:06:48.658 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:48.658 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:06:48.658 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:06:48.658 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:06:48.658 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:48.661 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:06:48.694 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:06:48.797 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:06:48.965 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:06:48.996 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:06:49.020 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:06:49.021 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:49.021 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:06:49.021 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:06:49.021 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:06:49.021 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:49.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:06:49.084 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:06:49.116 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:06:49.141 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:06:49.142 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:49.142 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:06:49.142 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:06:49.142 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:06:49.142 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:49.181 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:06:49.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:06:49.250 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:06:49.255 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:06:49.313 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:06:49.324 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:06:49.352 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:06:49.376 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:06:49.385 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:06:49.571 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:06:49.625 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:06:49.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:06:49.712 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:06:49.764 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:49.822 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:06:49.839 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:06:49.869 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:06:49.905 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:06:49.965 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:06:50.014 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:06:50.049 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:06:50.076 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:06:50.159 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:06:50.191 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:06:50.215 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:50.267 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:06:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:06:50.290 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:50.290 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:06:50.290 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:50.342 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:06:50.365 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:06:50.365 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:50.365 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:06:50.365 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:06:50.365 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:06:50.366 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:50.413 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:06:50.442 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:50.452 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:06:50.483 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:06:50.497 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:06:50.514 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:06:50.527 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:06:50.579 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:06:50.605 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:06:50.668 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:06:50.691 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:06:50.691 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:50.691 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:06:50.692 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:06:50.692 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:06:50.692 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:06:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:06:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:06:52.758 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:06:52.783 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:06:52.891 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:06:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:53.068 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:06:53.101 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:06:53.127 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:06:53.128 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:53.128 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:06:53.128 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:06:53.128 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:06:53.128 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:53.177 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:06:53.189 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:06:53.290 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:06:53.351 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:06:53.554 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:06:53.587 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:53.662 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:06:53.663 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:06:53.663 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:06:53.663 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:06:53.696 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:06:54.327 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:06:54.354 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:06:54.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:54.422 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:06:54.446 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:06:54.447 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:54.447 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:06:54.447 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:54.497 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:06:54.522 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:54.839 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:06:54.865 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:06:55.221 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:06:55.263 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:06:55.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:06:55.335 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:06:55.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:06:55.542 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:06:55.592 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:06:55.623 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:06:55.660 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:06:55.699 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:06:56.320 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:06:56.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:06:56.384 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:06:57.228 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:06:57.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:06:57.375 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:06:57.463 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:06:57.512 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:06:57.537 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:06:57.563 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:06:57.563 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:06:57.655 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:06:57.656 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:06:57.740 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:06:57.765 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:06:57.772 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:06:57.773 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:06:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:06:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:06:58.034 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:06:58.099 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:06:59.693 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:06:59.735 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:06:59.776 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:06:59.924 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:06:59.951 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:07:00.162 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:07:00.214 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:07:00.244 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:07:00.274 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:07:00.312 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:07:00.948 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:00.992 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:01.014 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:01.911 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:07:02.005 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:07:02.051 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:07:02.144 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:07:02.183 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:07:02.209 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:07:02.234 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:07:02.234 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:07:02.310 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:07:02.310 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:07:02.394 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:07:02.419 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:07:02.425 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:02.426 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:02.612 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:02.613 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:07:02.688 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:07:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:07:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:07:03.755 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:07:03.756 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:07:03.757 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:07:03.769 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:07:03.770 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:07:03.771 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:07:03.772 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:07:03.773 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:07:03.774 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:07:03.775 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:07:03.776 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:07:03.777 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:07:03.778 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:07:03.779 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:07:03.780 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:07:03.781 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:07:05.804 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:07:05.804 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-09-12T02:07:05.804 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-09-12T02:07:05.804 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-09-12T02:07:05.805 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-09-12T02:07:05.805 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-09-12T02:07:05.805 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-09-12T02:07:05.805 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-09-12T02:07:05.805 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:05.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:07:06.015 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:07:06.015 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:06.015 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:07:06.015 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:06.027 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-09-12T02:07:06.028 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.029 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.030 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:07:06.031 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.032 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:07:06.033 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:07:06.034 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:07:06.035 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:07:06.036 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:07:06.037 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:07:06.038 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:07:06.039 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:07:06.040 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:07:06.041 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:07:06.041 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:07:06.041 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:07:06.041 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-09-12T02:07:06.445 DEBUG:teuthology.parallel:result is None 2024-09-12T02:07:07.536 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:07:07.562 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:07:07.563 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:07.563 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:07:07.563 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:07:07.563 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:07:07.563 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:07.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:07:07.826 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:07:07.826 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:07.827 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:07:07.827 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:07:07.827 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:07:07.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:08.239 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:08.239 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:07:08.240 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:07:08.241 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:07:08.242 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:07:08.243 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:07:08.244 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:07:08.245 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:07:08.246 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:07:08.247 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:07:08.248 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:07:08.249 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:07:08.250 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:07:08.251 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:07:08.252 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:07:08.287 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:07:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:07:08.310 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:08.311 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:07:08.311 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:07:08.311 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:07:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:08.418 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:07:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:07:08.539 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:07:08.734 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:07:08.797 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:07:08.869 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:07:08.967 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:07:09.113 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:07:09.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:07:09.195 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:07:09.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:07:09.641 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:07:09.671 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:07:09.719 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:07:09.794 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:07:09.819 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:07:10.927 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:07:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:11.311 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:07:11.312 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.313 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:11.314 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.315 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.316 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:07:11.317 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:07:11.318 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:07:11.319 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:07:11.320 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:07:11.321 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:07:11.322 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:07:11.323 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:07:11.324 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:07:11.325 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:07:11.326 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:07:11.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:07:11.326 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-09-12T02:07:11.839 DEBUG:teuthology.parallel:result is None 2024-09-12T02:07:11.978 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:07:12.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:07:12.115 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:07:12.284 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:07:12.314 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:07:12.338 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:07:12.338 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:12.338 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:07:12.338 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:07:12.338 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:07:12.339 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:12.399 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:07:12.410 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:07:12.512 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:07:12.565 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:07:13.540 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:07:13.576 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:07:13.599 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:07:13.599 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:13.599 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:07:13.600 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:07:13.600 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:07:13.600 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:07:13.666 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:07:13.666 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:13.666 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:07:13.666 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:13.727 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:07:13.751 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:14.070 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:07:14.093 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:07:14.093 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:07:14.093 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:07:14.093 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:07:14.094 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:07:14.094 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:18.907 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:07:18.948 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:07:18.990 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:07:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:07:19.048 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:07:19.244 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:07:19.295 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:07:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:07:19.355 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:07:19.392 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:07:20.025 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:20.065 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:07:20.998 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:07:21.086 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:07:21.128 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:07:21.213 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:07:21.256 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:07:21.281 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:07:21.307 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:07:21.307 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:07:21.391 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:07:21.475 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:07:21.500 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:07:21.506 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:21.506 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:21.693 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:07:21.694 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:07:21.769 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:07:21.836 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:07:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:07:27.286 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:07:27.287 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:07:27.299 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:07:27.299 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:07:27.300 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:07:27.301 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:07:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:07:27.303 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:07:27.304 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:07:27.305 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:07:27.306 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:07:27.307 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:07:27.308 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:07:27.309 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:07:27.310 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:07:27.311 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:07:30.245 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:07:30.245 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:30.245 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-09-12T02:07:30.245 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:30.245 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.246 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.247 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:07:30.248 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:07:30.249 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:07:30.250 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:07:30.251 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.252 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:07:30.253 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:07:30.254 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:07:30.255 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:07:30.256 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.257 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:07:30.258 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:07:30.259 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:07:30.259 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-09-12T02:07:30.663 DEBUG:teuthology.parallel:result is None 2024-09-12T02:07:30.663 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:30.841 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:07:30.869 INFO:teuthology.orchestra.run.smithi003.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:30.871 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:30.871 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:07:30.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:31.028 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:07:31.051 INFO:teuthology.orchestra.run.smithi026.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:31.052 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:31.052 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:07:31.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:31.201 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:07:31.228 INFO:teuthology.orchestra.run.smithi092.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:31.228 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:07:31.228 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:07:31.230 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T02:07:31.230 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:07:31.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:07:31.264 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:07:31.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:07:31.295 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:07:31.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:07:31.325 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T02:07:31.325 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:07:31.325 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:07:31.356 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:07:31.425 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:07:31.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:07:31.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:07:31.524 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:07:31.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:07:31.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:07:31.618 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T02:07:31.618 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:07:31.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:07:31.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:07:31.711 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:07:31.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:07:31.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:07:31.808 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:07:31.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:07:31.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:07:31.899 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T02:07:32.456 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:07:32.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:07:32.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:07:32.547 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:07:32.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:07:32.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:07:32.647 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:07:32.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:07:32.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:07:32.736 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T02:07:32.842 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:07:32.842 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:32.843 INFO:tasks.cephadm:Cluster fsid is c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:07:32.843 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T02:07:32.843 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.26', 'mon.c': '172.21.15.92'} 2024-09-12T02:07:32.843 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-09-12T02:07:32.844 INFO:tasks.cephadm:First mgr is x 2024-09-12T02:07:32.844 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T02:07:32.844 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-09-12T02:07:32.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2024-09-12T02:07:32.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-09-12T02:07:32.926 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T02:07:32.926 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:33.077 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T02:07:33.231 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:07:33.232 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:07:33.233 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:07:33.233 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:33.591 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:07 /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:33.592 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:33.967 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:07 /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:33.968 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:34.345 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:07 /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:34.345 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:34.365 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:34.385 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:07:34.421 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T02:07:34.422 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:07:34.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:07:34.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:07:36.014 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:07:36.022 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:07:36.082 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:08:02.436 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-12T02:08:02.437 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:08:29.880 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:08:29.881 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-09-12T02:08:30.493 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-12T02:08:30.519 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-09-12T02:08:30.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2024-09-12T02:08:30.588 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-09-12T02:08:30.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-09-12T02:08:30.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2024-09-12T02:08:30.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-09-12T02:08:30.697 INFO:tasks.cephadm:Writing seed config... 2024-09-12T02:08:30.698 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T02:08:30.698 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T02:08:30.698 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T02:08:30.698 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T02:08:30.699 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T02:08:30.700 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T02:08:30.701 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T02:08:30.702 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T02:08:30.703 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:08:30.703 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T02:08:30.721 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c595fa10-70ab-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T02:08:30.721 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T02:08:30.764 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T02:08:30.808 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T02:08:30.808 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:08:31.022 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:08:31.022 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'c595fa10-70ab-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-09-12T02:08:31.023 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T02:08:31.023 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-09-12T02:08:31.046 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:08:31.047 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-09-12T02:08:31.047 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-09-12T02:08:31.055 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:08:31.055 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:08:31.061 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:08:31.061 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:08:31.068 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-12T02:08:31.075 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-12T02:08:31.075 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:08:31.075 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-09-12T02:08:31.110 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:08:31.111 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T02:08:31.111 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-09-12T02:08:31.111 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-09-12T02:08:31.117 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:08:31.117 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:08:31.124 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:08:31.124 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:08:31.131 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-09-12T02:08:31.137 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-09-12T02:08:31.137 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:08:31.138 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-09-12T02:08:31.138 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:31.138 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 140130480059392 on /run/cephadm/c595fa10-70ab-11ef-bcea-c7b262605968.lock 2024-09-12T02:08:31.138 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 140130480059392 acquired on /run/cephadm/c595fa10-70ab-11ef-bcea-c7b262605968.lock 2024-09-12T02:08:31.138 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-09-12T02:08:31.139 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-09-12T02:08:31.139 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-12T02:08:31.142 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-09-12T02:08:31.142 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-09-12T02:08:31.144 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T02:08:31.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T02:08:31.145 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1764sec hoplimit 64 pref medium 2024-09-12T02:08:31.147 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T02:08:31.147 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T02:08:31.147 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:08:31.147 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T02:08:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T02:08:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:08:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-12T02:08:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-09-12T02:08:31.148 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T02:08:31.149 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T02:08:31.149 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:08:31.614 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:08:31.615 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T02:08:32.512 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:08:32.512 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:08:32.512 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T02:08:33.368 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-09-12T02:08:33.368 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-09-12T02:08:34.282 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAhTeJmE5joKhAAEmXJZsyKIIIsSpM6dU0wbA== 2024-09-12T02:08:35.278 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAiTeJmKg42JhAAWNK8P0dgQPIcFejFGJUoGA== 2024-09-12T02:08:36.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQAjTeJmGQeZIxAAA5Cf9wnbjv35aSJTGjJiJw== 2024-09-12T02:08:36.118 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:08:37.032 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-09-12T02:08:37.033 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:37.033 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:08:37.033 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:08:37.033 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-09-12T02:08:37.946 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-09-12T02:08:38.162 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T02:08:38.328 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T02:08:38.515 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target. 2024-09-12T02:08:38.515 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target. 2024-09-12T02:08:38.783 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:08:38.783 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T02:08:38.970 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target.wants/ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968@.service. 2024-09-12T02:08:39.024 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:38 smithi003 systemd[1]: Starting Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:08:39.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 podman[31090]: 2024-09-12 02:08:39.112760358 +0000 UTC m=+0.016296938 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:08:39.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 podman[31090]: 2024-09-12 02:08:39.212642715 +0000 UTC m=+0.116179285 container create b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T02:08:39.523 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:08:39.524 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-12T02:08:39.532 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:08:39.533 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:08:39.533 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-12T02:08:39.533 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:08:39.533 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-09-12T02:08:39.533 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-09-12T02:08:39.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 podman[31090]: 2024-09-12 02:08:39.423215387 +0000 UTC m=+0.326751958 container init b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T02:08:39.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 podman[31090]: 2024-09-12 02:08:39.426807278 +0000 UTC m=+0.330343849 container start b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:08:39.586 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 ceph-mon[31108]: mkfs c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:39.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 ceph-mon[31108]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:08:39.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 bash[31090]: b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d 2024-09-12T02:08:39.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:39 smithi003 systemd[1]: Started Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:08:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:08:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: monmap epoch 1 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: last_changed 2024-09-12T02:08:36.454578+0000 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: min_mon_release 19 (squid) 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: election_strategy: 1 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: fsmap 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: mgrmap e1: no daemons active 2024-09-12T02:08:40.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:40 smithi003 ceph-mon[31108]: from='client.? 172.21.15.3:0/679795187' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T02:08:43.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T02:08:43.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:43.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T02:08:43.515 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.563306s) 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:43.516 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.954+0000 7fb873bfa640 1 Processor -- start 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.954+0000 7fb873bfa640 1 -- start start 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb873bfa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb873bfa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c106fd0 con 0x7fb86c106600 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb871996640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb871996640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47292/0 (socket says 172.21.15.3:47292) 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb871996640 1 -- 172.21.15.3:0/4119073588 learned_addr learned my addr 172.21.15.3:0/4119073588 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.955+0000 7fb871996640 1 -- 172.21.15.3:0/4119073588 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c107800 con 0x7fb86c106600 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.957+0000 7fb871996640 1 --2- 172.21.15.3:0/4119073588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009e80 tx=0x7fb85c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89510e922c2311e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb870994640 1 -- 172.21.15.3:0/4119073588 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb85c036070 con 0x7fb86c106600 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb870994640 1 -- 172.21.15.3:0/4119073588 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c031400 con 0x7fb86c106600 2024-09-12T02:08:43.517 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb870994640 1 -- 172.21.15.3:0/4119073588 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb85c031700 con 0x7fb86c106600 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 -- 172.21.15.3:0/4119073588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 msgr2=0x7fb86c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 --2- 172.21.15.3:0/4119073588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009e80 tx=0x7fb85c02a7a0 comp rx=0 tx=0).stop 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 -- 172.21.15.3:0/4119073588 shutdown_connections 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 --2- 172.21.15.3:0/4119073588 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c106a00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 -- 172.21.15.3:0/4119073588 >> 172.21.15.3:0/4119073588 conn(0x7fb86c101e10 msgr2=0x7fb86c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 -- 172.21.15.3:0/4119073588 shutdown_connections 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.958+0000 7fb873bfa640 1 -- 172.21.15.3:0/4119073588 wait complete. 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb873bfa640 1 Processor -- start 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb873bfa640 1 -- start start 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb873bfa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb873bfa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c107da0 con 0x7fb86c106600 2024-09-12T02:08:43.518 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb871996640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb871996640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47302/0 (socket says 172.21.15.3:47302) 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.959+0000 7fb871996640 1 -- 172.21.15.3:0/679795187 learned_addr learned my addr 172.21.15.3:0/679795187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb871996640 1 -- 172.21.15.3:0/679795187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c19b460 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb871996640 1 --2- 172.21.15.3:0/679795187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb85c034040 tx=0x7fb85c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb85c041070 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c03c650 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb85c036040 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c19b6f0 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c19e3e0 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb85c044070 con 0x7fb86c106600 2024-09-12T02:08:43.519 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.960+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb85c046c10 con 0x7fb86c106600 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.961+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c1083f0 con 0x7fb86c106600 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:39.964+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb85c03c950 con 0x7fb86c106600 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.025+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb86c19e980 con 0x7fb86c106600 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb86affd640 1 -- 172.21.15.3:0/679795187 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb85c03cb30 con 0x7fb86c106600 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 msgr2=0x7fb86c19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb873bfa640 1 --2- 172.21.15.3:0/679795187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb85c034040 tx=0x7fb85c004a40 comp rx=0 tx=0).stop 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 shutdown_connections 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb873bfa640 1 --2- 172.21.15.3:0/679795187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb86c106600 0x7fb86c19af20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.026+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 >> 172.21.15.3:0/679795187 conn(0x7fb86c101e10 msgr2=0x7fb86c07b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.027+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 shutdown_connections 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:40.027+0000 7fb873bfa640 1 -- 172.21.15.3:0/679795187 wait complete. 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-09-12T02:08:43.520 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T02:08:46.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:46 smithi003 ceph-mon[31108]: from='client.? 172.21.15.3:0/19310681' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:08:46.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:46 smithi003 ceph-mon[31108]: from='client.? 172.21.15.3:0/19310681' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T02:08:52.003 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:52.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:08:52.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:52.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-12T02:08:52.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:08:52.004 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:08:52.005 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f877e9e7640 1 Processor -- start 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f877e9e7640 1 -- start start 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f877e9e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f877e9e7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8778108130 con 0x7f8778107760 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f8777fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f8777fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47304/0 (socket says 172.21.15.3:47304) 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.584+0000 7f8777fff640 1 -- 172.21.15.3:0/3422453055 learned_addr learned my addr 172.21.15.3:0/3422453055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.585+0000 7f8777fff640 1 -- 172.21.15.3:0/3422453055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8778108960 con 0x7f8778107760 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.585+0000 7f8777fff640 1 --2- 172.21.15.3:0/3422453055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8768009820 tx=0x7f876802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722a46382228b189 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f8776ffd640 1 -- 172.21.15.3:0/3422453055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768037070 con 0x7f8778107760 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f8776ffd640 1 -- 172.21.15.3:0/3422453055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768032440 con 0x7f8778107760 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f8776ffd640 1 -- 172.21.15.3:0/3422453055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768032740 con 0x7f8778107760 2024-09-12T02:08:52.006 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 -- 172.21.15.3:0/3422453055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 msgr2=0x7f8778107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 --2- 172.21.15.3:0/3422453055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8768009820 tx=0x7f876802a270 comp rx=0 tx=0).stop 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 -- 172.21.15.3:0/3422453055 shutdown_connections 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 --2- 172.21.15.3:0/3422453055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f8778107b60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 -- 172.21.15.3:0/3422453055 >> 172.21.15.3:0/3422453055 conn(0x7f8778102ff0 msgr2=0x7f8778105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 -- 172.21.15.3:0/3422453055 shutdown_connections 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.586+0000 7f877e9e7640 1 -- 172.21.15.3:0/3422453055 wait complete. 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.587+0000 7f877e9e7640 1 Processor -- start 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.587+0000 7f877e9e7640 1 -- start start 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.587+0000 7f877e9e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.587+0000 7f877e9e7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8778108f00 con 0x7f8778107760 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f8777fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:52.007 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f8777fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47310/0 (socket says 172.21.15.3:47310) 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f8777fff640 1 -- 172.21.15.3:0/19310681 learned_addr learned my addr 172.21.15.3:0/19310681 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f8777fff640 1 -- 172.21.15.3:0/19310681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877819b460 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f8777fff640 1 --2- 172.21.15.3:0/19310681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f87680090b0 tx=0x7f87680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768041070 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876803c7a0 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877819b6f0 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768037040 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.588+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877819e3e0 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.589+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8768044070 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.589+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f876803d3f0 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.590+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8778109550 con 0x7f8778107760 2024-09-12T02:08:52.008 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.592+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8768046450 con 0x7f8778107760 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.653+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f877819fab0 con 0x7f8778107760 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.656+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+452 (secure 0 0 0) 0x7f8768046630 con 0x7f8778107760 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.657+0000 7f87757fa640 1 -- 172.21.15.3:0/19310681 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8768056070 con 0x7f8778107760 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.657+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 msgr2=0x7f877819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.657+0000 7f877e9e7640 1 --2- 172.21.15.3:0/19310681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f87680090b0 tx=0x7f87680046c0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.658+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 shutdown_connections 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.658+0000 7f877e9e7640 1 --2- 172.21.15.3:0/19310681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8778107760 0x7f877819af20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.658+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 >> 172.21.15.3:0/19310681 conn(0x7f8778102ff0 msgr2=0x7f877818f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.658+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 shutdown_connections 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:46.658+0000 7f877e9e7640 1 -- 172.21.15.3:0/19310681 wait complete. 2024-09-12T02:08:52.009 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-09-12T02:08:52.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:52 smithi003 ceph-mon[31108]: from='client.? 172.21.15.3:0/2287313066' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.440+0000 7f9f8538f640 1 Processor -- start 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.440+0000 7f9f8538f640 1 -- start start 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.440+0000 7f9f8538f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.441+0000 7f9f8538f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8007bdd0 con 0x7f9f801081f0 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.441+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.441+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46960/0 (socket says 172.21.15.3:46960) 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.441+0000 7f9f7effd640 1 -- 172.21.15.3:0/1958439978 learned_addr learned my addr 172.21.15.3:0/1958439978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.441+0000 7f9f7effd640 1 -- 172.21.15.3:0/1958439978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8010ab20 con 0x7f9f801081f0 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.442+0000 7f9f7effd640 1 --2- 172.21.15.3:0/1958439978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009820 tx=0x7f9f7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f7d5d0140dfce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.017 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.442+0000 7f9f7e7fc640 1 -- 172.21.15.3:0/1958439978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70037070 con 0x7f9f801081f0 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.442+0000 7f9f7e7fc640 1 -- 172.21.15.3:0/1958439978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f70032900 con 0x7f9f801081f0 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.442+0000 7f9f7e7fc640 1 -- 172.21.15.3:0/1958439978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70030850 con 0x7f9f801081f0 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- 172.21.15.3:0/1958439978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 msgr2=0x7f9f8010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 --2- 172.21.15.3:0/1958439978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009820 tx=0x7f9f7002a270 comp rx=0 tx=0).stop 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- 172.21.15.3:0/1958439978 shutdown_connections 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 --2- 172.21.15.3:0/1958439978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8010a5e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- 172.21.15.3:0/1958439978 >> 172.21.15.3:0/1958439978 conn(0x7f9f80101e10 msgr2=0x7f9f80104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- 172.21.15.3:0/1958439978 shutdown_connections 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- 172.21.15.3:0/1958439978 wait complete. 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 Processor -- start 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.443+0000 7f9f8538f640 1 -- start start 2024-09-12T02:08:53.018 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f8538f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f8538f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8010bb00 con 0x7f9f801081f0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46972/0 (socket says 172.21.15.3:46972) 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7effd640 1 -- 172.21.15.3:0/2287313066 learned_addr learned my addr 172.21.15.3:0/2287313066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7effd640 1 -- 172.21.15.3:0/2287313066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8019def0 con 0x7f9f801081f0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7effd640 1 --2- 172.21.15.3:0/2287313066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9f70031000 tx=0x7f9f700326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70042070 con 0x7f9f801081f0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f70032be0 con 0x7f9f801081f0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70037040 con 0x7f9f801081f0 2024-09-12T02:08:53.019 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.444+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8019e180 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.445+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f801a0e70 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.445+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f70045070 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.445+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f70047b20 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.446+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f80108650 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.448+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9f7004c030 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.509+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9f801a1040 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.509+0000 7f9f7cff9640 1 -- 172.21.15.3:0/2287313066 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f9f700328c0 con 0x7f9f801081f0 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 msgr2=0x7f9f8019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 --2- 172.21.15.3:0/2287313066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9f70031000 tx=0x7f9f700326e0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 shutdown_connections 2024-09-12T02:08:53.020 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 --2- 172.21.15.3:0/2287313066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f801081f0 0x7f9f8019d9b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:53.021 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 >> 172.21.15.3:0/2287313066 conn(0x7f9f80101e10 msgr2=0x7f9f80102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:53.021 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 shutdown_connections 2024-09-12T02:08:53.021 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:52.510+0000 7f9f8538f640 1 -- 172.21.15.3:0/2287313066 wait complete. 2024-09-12T02:08:53.021 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-09-12T02:08:53.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 systemd[1]: Stopping Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:08:53.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31104]: 2024-09-12T02:08:53.310+0000 7fbc39312640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:08:53.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31104]: 2024-09-12T02:08:53.310+0000 7fbc39312640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T02:08:53.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 podman[31354]: 2024-09-12 02:08:53.424890426 +0000 UTC m=+0.243410827 container died b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:08:54.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 podman[31354]: 2024-09-12 02:08:53.786578324 +0000 UTC m=+0.605098722 container cleanup b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:08:54.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:53 smithi003 bash[31354]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:08:54.404 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 podman[31368]: 2024-09-12 02:08:54.06248917 +0000 UTC m=+0.628012022 container remove b1928996f4014a9ef56caaca854271f765d84f68599e62eaf03ce32071f4fe8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T02:08:54.404 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:08:54.404 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 systemd[1]: Stopped Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:08:54.405 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 systemd[1]: Starting Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 podman[31458]: 2024-09-12 02:08:54.40488869 +0000 UTC m=+0.095920268 container create 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 podman[31458]: 2024-09-12 02:08:54.325471347 +0000 UTC m=+0.016502894 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 podman[31458]: 2024-09-12 02:08:54.607557249 +0000 UTC m=+0.298588786 container init 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 podman[31458]: 2024-09-12 02:08:54.610639973 +0000 UTC m=+0.301671509 container start 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: pidfile_write: ignore empty --pid-file 2024-09-12T02:08:54.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: load: jerasure load: lrc 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Git sha 0 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: DB SUMMARY 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: DB Session ID: FGQKTRTQHAQ4V4BTDR5U 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105812 ; 2024-09-12T02:08:54.708 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.env: 0x5570388b2940 2024-09-12T02:08:54.709 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.info_log: 0x55703a54d6e0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.statistics: (nil) 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.use_fsync: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:08:54.710 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.db_log_dir: 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.wal_dir: 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:08:54.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.write_buffer_manager: 0x55703a55cdc0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.unordered_write: 0 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:08:54.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.row_cache: None 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.wal_filter: None 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.wal_compression: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:08:54.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:08:54.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_open_files: -1 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:08:54.715 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Compression algorithms supported: 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kZSTD supported: 0 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:08:54.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.merge_operator: 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_filter: None 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55703a54d320) 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:08:54.717 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x55703a5411f0 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-09-12T02:08:54.718 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-09-12T02:08:54.719 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-09-12T02:08:54.720 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-09-12T02:08:54.720 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-09-12T02:08:54.720 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:08:54.721 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression: NoCompression 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.num_levels: 7 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:08:54.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:08:54.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:08:54.724 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:08:54.725 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:08:54.726 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.ttl: 2592000 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:08:54.727 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T02:08:54.728 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 648b5c10-b9f4-4b3d-bc75-513ccf606b69 2024-09-12T02:08:54.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726106934637594, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T02:08:54.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T02:08:54.729 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T02:08:54.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726106934638711, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100804, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98699, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14437, "raw_average_key_size": 49, "raw_value_size": 90900, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726106934, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "648b5c10-b9f4-4b3d-bc75-513ccf606b69", "db_session_id": "FGQKTRTQHAQ4V4BTDR5U", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:08:54.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726106934638810, "job": 1, "event": "recovery_finished"} 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55703a56ee00 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: DB pointer 0x55703a66a000 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:08:54.730 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 100.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 100.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:08:54.731 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 28.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 28.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:08:54.732 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x55703a5411f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???) e1 preinit fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).mds e1 new map 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).mds e1 print_map 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: btime 2024-09-12T02:08:39:464980+0000 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:08:54.733 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 ceph-mon[31476]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 bash[31458]: 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be 2024-09-12T02:08:54.734 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:54 smithi003 systemd[1]: Started Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:08:55.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:08:55.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: monmap epoch 1 2024-09-12T02:08:55.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:08:55.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: last_changed 2024-09-12T02:08:36.454578+0000 2024-09-12T02:08:55.722 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: min_mon_release 19 (squid) 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: election_strategy: 1 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: fsmap 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: mgrmap e1: no daemons active 2024-09-12T02:08:55.723 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:55 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/755641958' entity='client.admin' 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.067+0000 7fed5a578640 1 Processor -- start 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.067+0000 7fed5a578640 1 -- start start 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.067+0000 7fed5a578640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.068+0000 7fed5a578640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed54108190 con 0x7fed541077c0 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.068+0000 7fed53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.068+0000 7fed53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46986/0 (socket says 172.21.15.3:46986) 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.068+0000 7fed53fff640 1 -- 172.21.15.3:0/822328133 learned_addr learned my addr 172.21.15.3:0/822328133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:55.728 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.068+0000 7fed53fff640 1 -- 172.21.15.3:0/822328133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed54108920 con 0x7fed541077c0 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.070+0000 7fed53fff640 1 --2- 172.21.15.3:0/822328133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed44009820 tx=0x7fed4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdbb363c24ebb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.070+0000 7fed52ffd640 1 -- 172.21.15.3:0/822328133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44037070 con 0x7fed541077c0 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.070+0000 7fed52ffd640 1 -- 172.21.15.3:0/822328133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed44032900 con 0x7fed541077c0 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.070+0000 7fed52ffd640 1 -- 172.21.15.3:0/822328133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44030850 con 0x7fed541077c0 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- 172.21.15.3:0/822328133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 msgr2=0x7fed54107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 --2- 172.21.15.3:0/822328133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed44009820 tx=0x7fed4402a270 comp rx=0 tx=0).stop 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- 172.21.15.3:0/822328133 shutdown_connections 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 --2- 172.21.15.3:0/822328133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed54107bc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- 172.21.15.3:0/822328133 >> 172.21.15.3:0/822328133 conn(0x7fed54102fb0 msgr2=0x7fed541053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- 172.21.15.3:0/822328133 shutdown_connections 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- 172.21.15.3:0/822328133 wait complete. 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 Processor -- start 2024-09-12T02:08:55.729 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.071+0000 7fed5a578640 1 -- start start 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed5a578640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed5a578640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fed5410bcd0 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed53fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46996/0 (socket says 172.21.15.3:46996) 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed53fff640 1 -- 172.21.15.3:0/755641958 learned_addr learned my addr 172.21.15.3:0/755641958 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed53fff640 1 -- 172.21.15.3:0/755641958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5419bcc0 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed53fff640 1 --2- 172.21.15.3:0/755641958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed44009940 tx=0x7fed44032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.072+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44042070 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed440308c0 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5419bf50 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44037040 con 0x7fed541077c0 2024-09-12T02:08:55.730 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5419ec40 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed44045070 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.073+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed44047bb0 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.074+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5410be50 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.076+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fed440327b0 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.138+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fed54107c40 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.140+0000 7fed517fa640 1 -- 172.21.15.3:0/755641958 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fed4403c760 con 0x7fed541077c0 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 msgr2=0x7fed5419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 --2- 172.21.15.3:0/755641958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed44009940 tx=0x7fed44032ce0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 shutdown_connections 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 --2- 172.21.15.3:0/755641958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fed541077c0 0x7fed5419b780 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 >> 172.21.15.3:0/755641958 conn(0x7fed54102fb0 msgr2=0x7fed54103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 shutdown_connections 2024-09-12T02:08:55.731 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:55.141+0000 7fed5a578640 1 -- 172.21.15.3:0/755641958 wait complete. 2024-09-12T02:08:55.732 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T02:08:55.754 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:08:55.755 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-09-12T02:08:55.755 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T02:08:55.755 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T02:08:56.010 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:08:56.010 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T02:08:56.167 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968.target.wants/ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c595fa10-70ab-11ef-bcea-c7b262605968@.service. 2024-09-12T02:08:56.277 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 systemd[1]: Starting Ceph mgr.x for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:08:56.643 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 podman[31686]: 2024-09-12 02:08:56.391161895 +0000 UTC m=+0.096650246 container create 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T02:08:56.643 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 podman[31686]: 2024-09-12 02:08:56.310848772 +0000 UTC m=+0.016337082 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:08:56.643 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 podman[31686]: 2024-09-12 02:08:56.568462316 +0000 UTC m=+0.273950633 container init 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T02:08:56.643 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 podman[31686]: 2024-09-12 02:08:56.571353494 +0000 UTC m=+0.276841808 container start 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T02:08:56.643 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 bash[31686]: 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 2024-09-12T02:08:56.657 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:08:56.657 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-12T02:08:56.665 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:08:56.665 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:08:56.665 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-12T02:08:56.665 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:08:56.674 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:08:56.674 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-09-12T02:08:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-09-12T02:08:56.897 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 systemd[1]: Started Ceph mgr.x for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:08:56.897 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:56.697+0000 7f812f9d6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:08:56.897 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:56.760+0000 7f812f9d6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:08:57.149 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:56.919+0000 7f812f9d6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:08:57.149 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:56 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:56.988+0000 7f812f9d6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:08:57.149 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:57.148+0000 7f812f9d6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:08:57.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2192415093' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:08:57.528 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:57.305+0000 7f812f9d6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:08:57.900 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:57.639+0000 7f812f9d6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:08:57.900 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:57.703+0000 7f812f9d6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:08:57.900 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:57 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:57.899+0000 7f812f9d6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:08:58.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:08:58.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:08:58.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:08:58.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:08:58.870 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:08:58.871 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.872 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:08:39:464980+0000", 2024-09-12T02:08:58.873 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.874 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:08:39.465647+0000", 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.092+0000 7ff722bca640 1 Processor -- start 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.092+0000 7ff722bca640 1 -- start start 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff722bca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff722bca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c07bdd0 con 0x7ff71c108200 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff720966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff720966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47028/0 (socket says 172.21.15.3:47028) 2024-09-12T02:08:58.875 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff720966640 1 -- 172.21.15.3:0/1982914665 learned_addr learned my addr 172.21.15.3:0/1982914665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.093+0000 7ff720966640 1 -- 172.21.15.3:0/1982914665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c10ab30 con 0x7ff71c108200 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.094+0000 7ff720966640 1 --2- 172.21.15.3:0/1982914665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff710009820 tx=0x7ff71002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50486330bfb0181d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.094+0000 7ff71b7fe640 1 -- 172.21.15.3:0/1982914665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff710037070 con 0x7ff71c108200 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.094+0000 7ff71b7fe640 1 -- 172.21.15.3:0/1982914665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff710032900 con 0x7ff71c108200 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.094+0000 7ff71b7fe640 1 -- 172.21.15.3:0/1982914665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff710030850 con 0x7ff71c108200 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 -- 172.21.15.3:0/1982914665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 msgr2=0x7ff71c10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 --2- 172.21.15.3:0/1982914665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff710009820 tx=0x7ff71002a270 comp rx=0 tx=0).stop 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 -- 172.21.15.3:0/1982914665 shutdown_connections 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 --2- 172.21.15.3:0/1982914665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c10a5f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 -- 172.21.15.3:0/1982914665 >> 172.21.15.3:0/1982914665 conn(0x7ff71c101e60 msgr2=0x7ff71c104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 -- 172.21.15.3:0/1982914665 shutdown_connections 2024-09-12T02:08:58.876 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.095+0000 7ff722bca640 1 -- 172.21.15.3:0/1982914665 wait complete. 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff722bca640 1 Processor -- start 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff722bca640 1 -- start start 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff722bca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff722bca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff71c10bb10 con 0x7ff71c108200 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff720966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff720966640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47042/0 (socket says 172.21.15.3:47042) 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff720966640 1 -- 172.21.15.3:0/2192415093 learned_addr learned my addr 172.21.15.3:0/2192415093 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.096+0000 7ff720966640 1 -- 172.21.15.3:0/2192415093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71c107fc0 con 0x7ff71c108200 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff720966640 1 --2- 172.21.15.3:0/2192415093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff710032ce0 tx=0x7ff710032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff710042070 con 0x7ff71c108200 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7100308c0 con 0x7ff71c108200 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71c1061b0 con 0x7ff71c108200 2024-09-12T02:08:58.877 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff710037040 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71c1066f0 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff710045070 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.097+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff710047b80 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.098+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71c108660 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.100+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff71003c720 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.163+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff71c106c20 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.163+0000 7ff719ffb640 1 -- 172.21.15.3:0/2192415093 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff71003c900 con 0x7ff71c108200 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 msgr2=0x7ff71c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 --2- 172.21.15.3:0/2192415093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff710032ce0 tx=0x7ff710032d10 comp rx=0 tx=0).stop 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 shutdown_connections 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 --2- 172.21.15.3:0/2192415093 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff71c108200 0x7ff71c107a80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 >> 172.21.15.3:0/2192415093 conn(0x7ff71c101e60 msgr2=0x7ff71c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:08:58.878 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 shutdown_connections 2024-09-12T02:08:58.879 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:08:57.164+0000 7ff722bca640 1 -- 172.21.15.3:0/2192415093 wait complete. 2024-09-12T02:08:58.879 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-09-12T02:08:58.909 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:58.837+0000 7f812f9d6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:08:58.909 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:58.908+0000 7f812f9d6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:08:59.934 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:59 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:59.663+0000 7f812f9d6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:08:59.934 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:59 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:59.783+0000 7f812f9d6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:09:00.204 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:08:59 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:59.932+0000 7f812f9d6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:09:00.204 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:08:59.998+0000 7f812f9d6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:09:00.204 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.157+0000 7f812f9d6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:09:00.467 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.230+0000 7f812f9d6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:09:00.468 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.294+0000 7f812f9d6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:09:00.468 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.465+0000 7f812f9d6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:09:00.758 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.535+0000 7f812f9d6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:09:00.758 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.606+0000 7f812f9d6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:09:00.758 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.694+0000 7f812f9d6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:09:00.758 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.757+0000 7f812f9d6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:09:01.027 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:00.819+0000 7f812f9d6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:09:01.687 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/760794367' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:09:01.971 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:01 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:01.685+0000 7f812f9d6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:09:01.973 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.983 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:09:01.984 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:09:01.985 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:08:39:464980+0000", 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:09:01.986 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:09:01.987 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:08:39.465647+0000", 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.255+0000 7fd98d711640 1 Processor -- start 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.255+0000 7fd98d711640 1 -- start start 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.255+0000 7fd98d711640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.255+0000 7fd98d711640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd98807c870 con 0x7fd98807de40 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.256+0000 7fd986ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.256+0000 7fd986ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51748/0 (socket says 172.21.15.3:51748) 2024-09-12T02:09:01.988 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.256+0000 7fd986ffd640 1 -- 172.21.15.3:0/2826696778 learned_addr learned my addr 172.21.15.3:0/2826696778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.256+0000 7fd986ffd640 1 -- 172.21.15.3:0/2826696778 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98807c9f0 con 0x7fd98807de40 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.257+0000 7fd986ffd640 1 --2- 172.21.15.3:0/2826696778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd978009820 tx=0x7fd97802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da85c61b1f0ec717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.257+0000 7fd985ffb640 1 -- 172.21.15.3:0/2826696778 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978037070 con 0x7fd98807de40 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.257+0000 7fd985ffb640 1 -- 172.21.15.3:0/2826696778 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd978032900 con 0x7fd98807de40 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.257+0000 7fd985ffb640 1 -- 172.21.15.3:0/2826696778 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978030850 con 0x7fd98807de40 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 -- 172.21.15.3:0/2826696778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 msgr2=0x7fd98807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 --2- 172.21.15.3:0/2826696778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd978009820 tx=0x7fd97802a270 comp rx=0 tx=0).stop 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 -- 172.21.15.3:0/2826696778 shutdown_connections 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 --2- 172.21.15.3:0/2826696778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98807c2a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 -- 172.21.15.3:0/2826696778 >> 172.21.15.3:0/2826696778 conn(0x7fd988102fa0 msgr2=0x7fd9881053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 -- 172.21.15.3:0/2826696778 shutdown_connections 2024-09-12T02:09:01.989 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.258+0000 7fd98d711640 1 -- 172.21.15.3:0/2826696778 wait complete. 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd98d711640 1 Processor -- start 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd98d711640 1 -- start start 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd98d711640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd98d711640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd98810dda0 con 0x7fd98807de40 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd986ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd986ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51760/0 (socket says 172.21.15.3:51760) 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd986ffd640 1 -- 172.21.15.3:0/760794367 learned_addr learned my addr 172.21.15.3:0/760794367 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.259+0000 7fd986ffd640 1 -- 172.21.15.3:0/760794367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98819de40 con 0x7fd98807de40 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd986ffd640 1 --2- 172.21.15.3:0/760794367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097f0 tx=0x7fd978030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd978037040 con 0x7fd98807de40 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd978040070 con 0x7fd98807de40 2024-09-12T02:09:01.990 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98819e0d0 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd97803a390 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881a0dc0 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd97803a630 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.260+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd9780459a0 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.261+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98807c320 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.263+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd978045460 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.326+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd9881a0f90 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.326+0000 7fd96ffff640 1 -- 172.21.15.3:0/760794367 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd97803b430 con 0x7fd98807de40 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 msgr2=0x7fd98819d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 --2- 172.21.15.3:0/760794367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9780097f0 tx=0x7fd978030d80 comp rx=0 tx=0).stop 2024-09-12T02:09:01.991 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 shutdown_connections 2024-09-12T02:09:01.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 --2- 172.21.15.3:0/760794367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd98807de40 0x7fd98819d900 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:01.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 >> 172.21.15.3:0/760794367 conn(0x7fd988102fa0 msgr2=0x7fd988103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:01.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 shutdown_connections 2024-09-12T02:09:01.992 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:01.327+0000 7fd98d711640 1 -- 172.21.15.3:0/760794367 wait complete. 2024-09-12T02:09:01.992 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: Activating manager daemon x 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: mgrmap e2: x(active, starting, since 0.00308786s) 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:09:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: Manager daemon x is now available 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' 2024-09-12T02:09:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:02 smithi003 ceph-mon[31476]: from='mgr.14100 172.21.15.3:0/1419547824' entity='mgr.x' 2024-09-12T02:09:04.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:03 smithi003 ceph-mon[31476]: mgrmap e3: x(active, since 1.00786s) 2024-09-12T02:09:05.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:04 smithi003 ceph-mon[31476]: mgrmap e4: x(active, since 2s) 2024-09-12T02:09:05.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:04 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2639000711' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.223 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.224 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:09:05.225 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:05.225 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:09:05.225 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:09:05.225 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:09:05.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:09:05.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:08:39:464980+0000", 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:09:05.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:08:39.465647+0000", 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.378+0000 7f083a2d4640 1 Processor -- start 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.378+0000 7f083a2d4640 1 -- start start 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f083a2d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f083a2d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f083407bdf0 con 0x7f08341081d0 2024-09-12T02:09:05.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f0833fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f0833fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51822/0 (socket says 172.21.15.3:51822) 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f0833fff640 1 -- 172.21.15.3:0/594802872 learned_addr learned my addr 172.21.15.3:0/594802872 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.379+0000 7f0833fff640 1 -- 172.21.15.3:0/594802872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083410ab00 con 0x7f08341081d0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.380+0000 7f0833fff640 1 --2- 172.21.15.3:0/594802872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0824009820 tx=0x7f082402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7385345a0e4ceec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.380+0000 7f0832ffd640 1 -- 172.21.15.3:0/594802872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0824037070 con 0x7f08341081d0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.380+0000 7f0832ffd640 1 -- 172.21.15.3:0/594802872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0824032900 con 0x7f08341081d0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.380+0000 7f0832ffd640 1 -- 172.21.15.3:0/594802872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0824030850 con 0x7f08341081d0 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 -- 172.21.15.3:0/594802872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 msgr2=0x7f083410a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 --2- 172.21.15.3:0/594802872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0824009820 tx=0x7f082402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 -- 172.21.15.3:0/594802872 shutdown_connections 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 --2- 172.21.15.3:0/594802872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f083410a5c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:05.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 -- 172.21.15.3:0/594802872 >> 172.21.15.3:0/594802872 conn(0x7f0834101e10 msgr2=0x7f0834104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 -- 172.21.15.3:0/594802872 shutdown_connections 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.381+0000 7f083a2d4640 1 -- 172.21.15.3:0/594802872 wait complete. 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f083a2d4640 1 Processor -- start 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f083a2d4640 1 -- start start 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f083a2d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f083a2d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f083410bb40 con 0x7f08341081d0 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f0833fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f0833fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51836/0 (socket says 172.21.15.3:51836) 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.382+0000 7f0833fff640 1 -- 172.21.15.3:0/2639000711 learned_addr learned my addr 172.21.15.3:0/2639000711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f0833fff640 1 -- 172.21.15.3:0/2639000711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0834107f90 con 0x7f08341081d0 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f0833fff640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08240097f0 tx=0x7f0824030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:05.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0824042070 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0824032ca0 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0834106180 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0824037040 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.383+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08341066c0 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.384+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0824045070 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.384+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0834108630 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.384+0000 7f08317fa640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 0x7f081c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.384+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f082406bf30 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.384+0000 7f08337fe640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 0x7f081c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.385+0000 7f08337fe640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 0x7f081c03b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f08280098c0 tx=0x7f0828006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.391+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f082406b950 con 0x7f08341081d0 2024-09-12T02:09:05.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.640+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0834106f00 con 0x7f08341081d0 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.640+0000 7f08317fa640 1 -- 172.21.15.3:0/2639000711 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f0824045520 con 0x7f08341081d0 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 msgr2=0x7f081c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 0x7f081c03b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f08280098c0 tx=0x7f0828006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 msgr2=0x7f0834107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f08240097f0 tx=0x7f0824030ec0 comp rx=0 tx=0).stop 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 shutdown_connections 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f081c038f20 0x7f081c03b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 --2- 172.21.15.3:0/2639000711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08341081d0 0x7f0834107a50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 >> 172.21.15.3:0/2639000711 conn(0x7f0834101e10 msgr2=0x7f0834102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 shutdown_connections 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:04.643+0000 7f083a2d4640 1 -- 172.21.15.3:0/2639000711 wait complete. 2024-09-12T02:09:05.239 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-09-12T02:09:06.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:05 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1309999514' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:09:06.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:09:06.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d69cf1640 1 Processor -- start 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d69cf1640 1 -- start start 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d69cf1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d69cf1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64108130 con 0x7f2d64107760 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51844/0 (socket says 172.21.15.3:51844) 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d637fe640 1 -- 172.21.15.3:0/2483627820 learned_addr learned my addr 172.21.15.3:0/2483627820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.595+0000 7f2d637fe640 1 -- 172.21.15.3:0/2483627820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d64108960 con 0x7f2d64107760 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.596+0000 7f2d637fe640 1 --2- 172.21.15.3:0/2483627820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009820 tx=0x7f2d5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4339e69f8fc19429 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.596+0000 7f2d627fc640 1 -- 172.21.15.3:0/2483627820 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d54037070 con 0x7f2d64107760 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.596+0000 7f2d627fc640 1 -- 172.21.15.3:0/2483627820 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d54032900 con 0x7f2d64107760 2024-09-12T02:09:06.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.596+0000 7f2d627fc640 1 -- 172.21.15.3:0/2483627820 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d54030850 con 0x7f2d64107760 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 -- 172.21.15.3:0/2483627820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 msgr2=0x7f2d64107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/2483627820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009820 tx=0x7f2d5402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 -- 172.21.15.3:0/2483627820 shutdown_connections 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/2483627820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d64107b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 -- 172.21.15.3:0/2483627820 >> 172.21.15.3:0/2483627820 conn(0x7f2d64102ff0 msgr2=0x7f2d64105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 -- 172.21.15.3:0/2483627820 shutdown_connections 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 -- 172.21.15.3:0/2483627820 wait complete. 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.597+0000 7f2d69cf1640 1 Processor -- start 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d69cf1640 1 -- start start 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d69cf1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d69cf1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6410bd10 con 0x7f2d64107760 2024-09-12T02:09:06.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51854/0 (socket says 172.21.15.3:51854) 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d637fe640 1 -- 172.21.15.3:0/1309999514 learned_addr learned my addr 172.21.15.3:0/1309999514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d637fe640 1 -- 172.21.15.3:0/1309999514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6419bcd0 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d637fe640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d540097f0 tx=0x7f2d54030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.598+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d54037040 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.599+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d54040070 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.599+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6419bf60 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.599+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d5403a390 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.599+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6419ec50 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f2d5403a630 con 0x7f2d64107760 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d60ff9640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 0x7f2d4c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:06.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6410be90 con 0x7f2d64107760 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d5406c220 con 0x7f2d64107760 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d62ffd640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 0x7f2d4c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.600+0000 7f2d62ffd640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 0x7f2d4c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d580098c0 tx=0x7f2d58006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.606+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d5403b430 con 0x7f2d64107760 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.781+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2d64107be0 con 0x7f2d64107760 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.782+0000 7f2d60ff9640 1 -- 172.21.15.3:0/1309999514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+452 (secure 0 0 0) 0x7f2d5406bf80 con 0x7f2d64107760 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 msgr2=0x7f2d4c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 0x7f2d4c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d580098c0 tx=0x7f2d58006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 msgr2=0x7f2d6419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2d540097f0 tx=0x7f2d54030d80 comp rx=0 tx=0).stop 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 shutdown_connections 2024-09-12T02:09:06.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f2d4c038f20 0x7f2d4c03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 --2- 172.21.15.3:0/1309999514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d64107760 0x7f2d6419b790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 >> 172.21.15.3:0/1309999514 conn(0x7f2d64102ff0 msgr2=0x7f2d64103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 shutdown_connections 2024-09-12T02:09:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:05.784+0000 7f2d69cf1640 1 -- 172.21.15.3:0/1309999514 wait complete. 2024-09-12T02:09:06.349 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-09-12T02:09:07.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/813402983' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T02:09:07.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ignoring --setuser ceph since I am not root 2024-09-12T02:09:07.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ignoring --setgroup ceph since I am not root 2024-09-12T02:09:07.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.140+0000 7ff734633040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:09:07.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.201+0000 7ff734633040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882c43d640 1 Processor -- start 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882c43d640 1 -- start start 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882c43d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882c43d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f882410a360 con 0x7f8824109990 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882a1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.589 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882a1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51856/0 (socket says 172.21.15.3:51856) 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.713+0000 7f882a1d9640 1 -- 172.21.15.3:0/2805179500 learned_addr learned my addr 172.21.15.3:0/2805179500 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.714+0000 7f882a1d9640 1 -- 172.21.15.3:0/2805179500 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882410ab10 con 0x7f8824109990 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.714+0000 7f882a1d9640 1 --2- 172.21.15.3:0/2805179500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8820009820 tx=0x7f882002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca63af6c83989af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.714+0000 7f88291d7640 1 -- 172.21.15.3:0/2805179500 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820037070 con 0x7f8824109990 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f88291d7640 1 -- 172.21.15.3:0/2805179500 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8820032900 con 0x7f8824109990 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f88291d7640 1 -- 172.21.15.3:0/2805179500 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820030850 con 0x7f8824109990 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 -- 172.21.15.3:0/2805179500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 msgr2=0x7f8824109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 --2- 172.21.15.3:0/2805179500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8820009820 tx=0x7f882002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 -- 172.21.15.3:0/2805179500 shutdown_connections 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 --2- 172.21.15.3:0/2805179500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f8824109d90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 -- 172.21.15.3:0/2805179500 >> 172.21.15.3:0/2805179500 conn(0x7f882407cf80 msgr2=0x7f882407d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.590 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 -- 172.21.15.3:0/2805179500 shutdown_connections 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.715+0000 7f882c43d640 1 -- 172.21.15.3:0/2805179500 wait complete. 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882c43d640 1 Processor -- start 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882c43d640 1 -- start start 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882c43d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882c43d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f882410dec0 con 0x7f8824109990 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882a1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882a1d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51868/0 (socket says 172.21.15.3:51868) 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882a1d9640 1 -- 172.21.15.3:0/813402983 learned_addr learned my addr 172.21.15.3:0/813402983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.716+0000 7f882a1d9640 1 -- 172.21.15.3:0/813402983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882419ded0 con 0x7f8824109990 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f882a1d9640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f88200305a0 tx=0x7f88200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820042070 con 0x7f8824109990 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8820032c60 con 0x7f8824109990 2024-09-12T02:09:07.591 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882419e160 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820037040 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.717+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a0e50 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.718+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8820045070 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.718+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882410e040 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.718+0000 7f881b7fe640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 0x7f881003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.718+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f882006bfc0 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.718+0000 7f88299d8640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 0x7f881003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.719+0000 7f88299d8640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 0x7f881003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88140098c0 tx=0x7f8814006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.723+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f882004b170 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.944+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f88241a14e0 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.995+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8820045c40 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.997+0000 7f881b7fe640 1 -- 172.21.15.3:0/813402983 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f882002fb70 con 0x7f8824109990 2024-09-12T02:09:07.592 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 msgr2=0x7f881003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 0x7f881003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88140098c0 tx=0x7f8814006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 msgr2=0x7f882419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f88200305a0 tx=0x7f88200327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 shutdown_connections 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f8810038f70 0x7f881003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 --2- 172.21.15.3:0/813402983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8824109990 0x7f882419d990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 >> 172.21.15.3:0/813402983 conn(0x7f882407cf80 msgr2=0x7f88241074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 shutdown_connections 2024-09-12T02:09:07.593 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:06.998+0000 7f882c43d640 1 -- 172.21.15.3:0/813402983 wait complete. 2024-09-12T02:09:07.707 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.357+0000 7ff734633040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:09:07.707 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.427+0000 7ff734633040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:09:07.707 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.584+0000 7ff734633040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:09:07.981 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:07 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:07.742+0000 7ff734633040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:09:08.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:08 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/813402983' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T02:09:08.234 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:08 smithi003 ceph-mon[31476]: mgrmap e5: x(active, since 5s) 2024-09-12T02:09:08.234 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:08 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:08.080+0000 7ff734633040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:09:08.234 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:08 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:08.144+0000 7ff734633040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:09:08.512 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:08 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:08.344+0000 7ff734633040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:09:08.800 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed6e35640 1 Processor -- start 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed6e35640 1 -- start start 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed6e35640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed6e35640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed007be50 con 0x7f7ed0105f60 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed4bd1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:08.801 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed4bd1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51904/0 (socket says 172.21.15.3:51904) 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.981+0000 7f7ed4bd1640 1 -- 172.21.15.3:0/3136589238 learned_addr learned my addr 172.21.15.3:0/3136589238 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.982+0000 7f7ed4bd1640 1 -- 172.21.15.3:0/3136589238 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01068a0 con 0x7f7ed0105f60 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.983+0000 7f7ed4bd1640 1 --2- 172.21.15.3:0/3136589238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009820 tx=0x7f7ec402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42249ab7d9c1e9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.983+0000 7f7ecf7fe640 1 -- 172.21.15.3:0/3136589238 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec4037070 con 0x7f7ed0105f60 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.983+0000 7f7ecf7fe640 1 -- 172.21.15.3:0/3136589238 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ec4032900 con 0x7f7ed0105f60 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.983+0000 7f7ecf7fe640 1 -- 172.21.15.3:0/3136589238 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec4030850 con 0x7f7ed0105f60 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 -- 172.21.15.3:0/3136589238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 msgr2=0x7f7ed0106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/3136589238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009820 tx=0x7f7ec402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 -- 172.21.15.3:0/3136589238 shutdown_connections 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/3136589238 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed0106360 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 -- 172.21.15.3:0/3136589238 >> 172.21.15.3:0/3136589238 conn(0x7f7ed0101e10 msgr2=0x7f7ed0104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:08.802 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 -- 172.21.15.3:0/3136589238 shutdown_connections 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 -- 172.21.15.3:0/3136589238 wait complete. 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.984+0000 7f7ed6e35640 1 Processor -- start 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed6e35640 1 -- start start 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed6e35640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed6e35640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed01079a0 con 0x7f7ed0105f60 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed4bd1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed4bd1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51916/0 (socket says 172.21.15.3:51916) 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed4bd1640 1 -- 172.21.15.3:0/1037721026 learned_addr learned my addr 172.21.15.3:0/1037721026 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed4bd1640 1 -- 172.21.15.3:0/1037721026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed019dec0 con 0x7f7ed0105f60 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ed4bd1640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40097f0 tx=0x7f7ec4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec4042070 con 0x7f7ed0105f60 2024-09-12T02:09:08.803 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.985+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ec4032ca0 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.986+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed019e150 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.986+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ec4037040 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.986+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a0e40 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7ec4045070 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed0106360 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ecdffb640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 0x7f7ec003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ec406bf60 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ecffff640 1 -- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 msgr2=0x7f7ec003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.987+0000 7f7ecffff640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 0x7f7ec003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:07.992+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ec406b980 con 0x7f7ed0105f60 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.188+0000 7f7ecffff640 1 -- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 msgr2=0x7f7ec003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:08.804 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.188+0000 7f7ecffff640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 0x7f7ec003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.221+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7ed002da40 con 0x7f7ed0105f60 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.222+0000 7f7ecdffb640 1 -- 172.21.15.3:0/1037721026 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f7ec4045520 con 0x7f7ed0105f60 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 msgr2=0x7f7ec003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 0x7f7ec003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 msgr2=0x7f7ed019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ec40097f0 tx=0x7f7ec4030ec0 comp rx=0 tx=0).stop 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 shutdown_connections 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f7ec0038fc0 0x7f7ec003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 --2- 172.21.15.3:0/1037721026 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ed0105f60 0x7f7ed019d980 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.224+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 >> 172.21.15.3:0/1037721026 conn(0x7f7ed0101e10 msgr2=0x7f7ed0194650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.225+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 shutdown_connections 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:08.225+0000 7f7ed6e35640 1 -- 172.21.15.3:0/1037721026 wait complete. 2024-09-12T02:09:08.805 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-12T02:09:08.806 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-09-12T02:09:09.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:09 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1037721026' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:09:09.361 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:09 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:09.290+0000 7ff734633040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:09:09.361 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:09 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:09.361+0000 7ff734633040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:09:10.395 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.123+0000 7ff734633040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:09:10.395 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.245+0000 7ff734633040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:09:10.691 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.395+0000 7ff734633040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:09:10.691 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.460+0000 7ff734633040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:09:10.691 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.617+0000 7ff734633040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:09:10.691 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.691+0000 7ff734633040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:09:10.990 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.755+0000 7ff734633040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:09:10.990 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.921+0000 7ff734633040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:09:10.990 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:10 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:10.990+0000 7ff734633040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:09:11.275 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:11 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:11.060+0000 7ff734633040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:09:11.275 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:11 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:11.147+0000 7ff734633040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:09:11.275 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:11 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:11.210+0000 7ff734633040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:09:11.275 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:11 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:11.274+0000 7ff734633040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:09:12.527 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:12.113+0000 7ff734633040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: Active manager daemon x restarted 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: Activating manager daemon x 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: mgrmap e6: x(active, starting, since 0.00387273s) 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:09:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: Manager daemon x is now available 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:09:12.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:12 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:09:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T02:09:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:09:13.678 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.188+0000 7f08ddb24640 1 Processor -- start 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08ddb24640 1 -- start start 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08ddb24640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08ddb24640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08d807be50 con 0x7f08d8108170 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48764/0 (socket says 172.21.15.3:48764) 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08d77fe640 1 -- 172.21.15.3:0/1363427071 learned_addr learned my addr 172.21.15.3:0/1363427071 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.189+0000 7f08d77fe640 1 -- 172.21.15.3:0/1363427071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d810aaa0 con 0x7f08d8108170 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.190+0000 7f08d77fe640 1 --2- 172.21.15.3:0/1363427071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009820 tx=0x7f08c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8577e0c36cad5776 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.190+0000 7f08d6ffd640 1 -- 172.21.15.3:0/1363427071 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8037070 con 0x7f08d8108170 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.190+0000 7f08d6ffd640 1 -- 172.21.15.3:0/1363427071 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08c8032900 con 0x7f08d8108170 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.190+0000 7f08d6ffd640 1 -- 172.21.15.3:0/1363427071 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8030850 con 0x7f08d8108170 2024-09-12T02:09:13.679 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 -- 172.21.15.3:0/1363427071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 msgr2=0x7f08d810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 --2- 172.21.15.3:0/1363427071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009820 tx=0x7f08c802a270 comp rx=0 tx=0).stop 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 -- 172.21.15.3:0/1363427071 shutdown_connections 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 --2- 172.21.15.3:0/1363427071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d810a560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 -- 172.21.15.3:0/1363427071 >> 172.21.15.3:0/1363427071 conn(0x7f08d8101e10 msgr2=0x7f08d8104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 -- 172.21.15.3:0/1363427071 shutdown_connections 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.191+0000 7f08ddb24640 1 -- 172.21.15.3:0/1363427071 wait complete. 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08ddb24640 1 Processor -- start 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08ddb24640 1 -- start start 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08ddb24640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08ddb24640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08d810bb40 con 0x7f08d8108170 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48772/0 (socket says 172.21.15.3:48772) 2024-09-12T02:09:13.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.192+0000 7f08d77fe640 1 -- 172.21.15.3:0/2004464797 learned_addr learned my addr 172.21.15.3:0/2004464797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08d77fe640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d8107f30 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08d77fe640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08c8031000 tx=0x7f08c80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8042070 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08c8032be0 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d8106120 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08c8037040 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.193+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d8106660 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f08c8045070 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08d57fa640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08d81085d0 con 0x7f08c4038fc0 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f08c806d3b0 con 0x7f08d8108170 2024-09-12T02:09:13.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08ceffd640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 msgr2=0x7f08c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.194+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.395+0000 7f08ceffd640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 msgr2=0x7f08c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.395+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.796+0000 7f08ceffd640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 msgr2=0x7f08c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:09.796+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:10.597+0000 7f08ceffd640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 msgr2=0x7f08c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/814335086 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:10.597+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:12.117+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f08c806be70 con 0x7f08d8108170 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:12.117+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 msgr2=0x7f08c403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:12.117+0000 7f08d57fa640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.118+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f08c80475d0 con 0x7f08d8108170 2024-09-12T02:09:13.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.118+0000 7f08d57fa640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 0x7f08c403eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.118+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08c8044e50 con 0x7f08c403c700 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.119+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 0x7f08c403eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.119+0000 7f08ceffd640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 0x7f08c403eaf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003980 tx=0x7f08c00092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.120+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f08c8044e50 con 0x7f08c403c700 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f08d8103190 con 0x7f08c403c700 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08d57fa640 1 -- 172.21.15.3:0/2004464797 <== mgr.14118 v2:172.21.15.3:6800/1539065682 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f08d8103190 con 0x7f08c403c700 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 msgr2=0x7f08c403eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 0x7f08c403eaf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f08c0003980 tx=0x7f08c00092b0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 msgr2=0x7f08d81079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08c8031000 tx=0x7f08c80326e0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 shutdown_connections 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f08c403c700 0x7f08c403eaf0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:6800/814335086,v1:172.21.15.3:6801/814335086] conn(0x7f08c4038fc0 0x7f08c403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 --2- 172.21.15.3:0/2004464797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f08d8108170 0x7f08d81079f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.127+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 >> 172.21.15.3:0/2004464797 conn(0x7f08d8101e10 msgr2=0x7f08d8102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.128+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 shutdown_connections 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:13.128+0000 7f08ddb24640 1 -- 172.21.15.3:0/2004464797 wait complete. 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-09-12T02:09:13.684 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-09-12T02:09:13.957 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:13 smithi003 ceph-mon[31476]: Found migration_current of "None". Setting to last migration. 2024-09-12T02:09:13.957 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:13 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:13.958 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:13 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:13.958 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:13 smithi003 ceph-mon[31476]: mgrmap e7: x(active, since 1.00531s) 2024-09-12T02:09:14.528 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:14.252+0000 7ff6a1198640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:09:14] ENGINE Error in HTTPServer.serve 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: Traceback (most recent call last): 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._connections.run(self.expiration_interval) 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._run(expiration_interval) 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: s = self.context.wrap_socket( 2024-09-12T02:09:14.529 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: return self.sslsocket_class._create( 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self.do_handshake() 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._sslobj.do_handshake() 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:09:14.530 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:14.927 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.038+0000 7fada98d2640 1 Processor -- start 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.038+0000 7fada98d2640 1 -- start start 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.038+0000 7fada98d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.038+0000 7fada98d2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fada407bdd0 con 0x7fada41081f0 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.039+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.039+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48854/0 (socket says 172.21.15.3:48854) 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.039+0000 7fada2ffd640 1 -- 172.21.15.3:0/3323848881 learned_addr learned my addr 172.21.15.3:0/3323848881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.039+0000 7fada2ffd640 1 -- 172.21.15.3:0/3323848881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada410ab20 con 0x7fada41081f0 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada2ffd640 1 --2- 172.21.15.3:0/3323848881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad94009820 tx=0x7fad9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeea3aa04bf23bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada27fc640 1 -- 172.21.15.3:0/3323848881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad94037070 con 0x7fada41081f0 2024-09-12T02:09:14.928 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada27fc640 1 -- 172.21.15.3:0/3323848881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad94032900 con 0x7fada41081f0 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada27fc640 1 -- 172.21.15.3:0/3323848881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad94030850 con 0x7fada41081f0 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada98d2640 1 -- 172.21.15.3:0/3323848881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 msgr2=0x7fada410a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.040+0000 7fada98d2640 1 --2- 172.21.15.3:0/3323848881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fad94009820 tx=0x7fad9402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 -- 172.21.15.3:0/3323848881 shutdown_connections 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 --2- 172.21.15.3:0/3323848881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada410a5e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 -- 172.21.15.3:0/3323848881 >> 172.21.15.3:0/3323848881 conn(0x7fada4101e10 msgr2=0x7fada4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 -- 172.21.15.3:0/3323848881 shutdown_connections 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 -- 172.21.15.3:0/3323848881 wait complete. 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 Processor -- start 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.041+0000 7fada98d2640 1 -- start start 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada98d2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:14.929 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada98d2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fada410bb00 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48870/0 (socket says 172.21.15.3:48870) 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada2ffd640 1 -- 172.21.15.3:0/690471450 learned_addr learned my addr 172.21.15.3:0/690471450 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada2ffd640 1 -- 172.21.15.3:0/690471450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada419bd00 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada2ffd640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad94031000 tx=0x7fad940326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad94042070 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad94032be0 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada419bf90 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad94037040 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.042+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada419ec80 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.043+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fad94045070 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.043+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada4108650 con 0x7fada41081f0 2024-09-12T02:09:14.930 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.044+0000 7fada88d0640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 0x7fad9003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.044+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad9406bd40 con 0x7fada41081f0 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.044+0000 7fad9bfff640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 0x7fad9003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.044+0000 7fad9bfff640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 0x7fad9003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0098c0 tx=0x7fad8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.053+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad9406f030 con 0x7fada41081f0 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.246+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fada41031a0 con 0x7fad90038e60 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.251+0000 7fada88d0640 1 -- 172.21.15.3:0/690471450 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fada41031a0 con 0x7fad90038e60 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 msgr2=0x7fad9003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 0x7fad9003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0098c0 tx=0x7fad8c006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 msgr2=0x7fada419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fad94031000 tx=0x7fad940326e0 comp rx=0 tx=0).stop 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 shutdown_connections 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fad90038e60 0x7fad9003b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:14.931 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 --2- 172.21.15.3:0/690471450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada41081f0 0x7fada419b7c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:14.932 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 >> 172.21.15.3:0/690471450 conn(0x7fada4101e10 msgr2=0x7fada4102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:14.932 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 shutdown_connections 2024-09-12T02:09:14.932 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:14.254+0000 7fada98d2640 1 -- 172.21.15.3:0/690471450 wait complete. 2024-09-12T02:09:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-mon[31476]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:09:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-mon[31476]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:09:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:14 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:14] ENGINE Bus STARTING 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:14] ENGINE Serving on https://172.21.15.3:7150 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:14] ENGINE Error in HTTPServer.serve 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:09:16.028 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-12T02:09:16.029 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:09:16.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-12T02:09:16.030 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:09:16.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:14] ENGINE Serving on http://172.21.15.3:8765 2024-09-12T02:09:16.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:14] ENGINE Bus STARTED 2024-09-12T02:09:16.030 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:15 smithi003 ceph-mon[31476]: mgrmap e8: x(active, since 2s) 2024-09-12T02:09:16.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T02:09:16.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.309+0000 7fbc25390640 1 Processor -- start 2024-09-12T02:09:16.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.309+0000 7fbc25390640 1 -- start start 2024-09-12T02:09:16.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc25390640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc25390640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2007bdd0 con 0x7fbc201081f0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48886/0 (socket says 172.21.15.3:48886) 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc1effd640 1 -- 172.21.15.3:0/4203909967 learned_addr learned my addr 172.21.15.3:0/4203909967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.310+0000 7fbc1effd640 1 -- 172.21.15.3:0/4203909967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2010ab20 con 0x7fbc201081f0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.311+0000 7fbc1effd640 1 --2- 172.21.15.3:0/4203909967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009820 tx=0x7fbc1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10409b7bc27c08c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.311+0000 7fbc1dffb640 1 -- 172.21.15.3:0/4203909967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10037070 con 0x7fbc201081f0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.311+0000 7fbc1dffb640 1 -- 172.21.15.3:0/4203909967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc10032900 con 0x7fbc201081f0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.311+0000 7fbc1dffb640 1 -- 172.21.15.3:0/4203909967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10030850 con 0x7fbc201081f0 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 -- 172.21.15.3:0/4203909967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 msgr2=0x7fbc2010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:16.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 --2- 172.21.15.3:0/4203909967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009820 tx=0x7fbc1002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 -- 172.21.15.3:0/4203909967 shutdown_connections 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 --2- 172.21.15.3:0/4203909967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2010a5e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 -- 172.21.15.3:0/4203909967 >> 172.21.15.3:0/4203909967 conn(0x7fbc20101e10 msgr2=0x7fbc20104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 -- 172.21.15.3:0/4203909967 shutdown_connections 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 -- 172.21.15.3:0/4203909967 wait complete. 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.312+0000 7fbc25390640 1 Processor -- start 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc25390640 1 -- start start 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc25390640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc25390640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010bb00 con 0x7fbc201081f0 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc1effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48896/0 (socket says 172.21.15.3:48896) 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc1effd640 1 -- 172.21.15.3:0/57300529 learned_addr learned my addr 172.21.15.3:0/57300529 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:16.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc1effd640 1 -- 172.21.15.3:0/57300529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2019bd00 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.313+0000 7fbc1effd640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097f0 tx=0x7fbc10030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.314+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10042070 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.314+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc10032ca0 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.314+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc10037040 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.314+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2019bf90 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.314+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2019ec80 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbc10045070 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc20108650 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc17fff640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc1006bf50 con 0x7fbc201081f0 2024-09-12T02:09:16.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc1e7fc640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.315+0000 7fbc1e7fc640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 0x7fbc0c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc080098c0 tx=0x7fbc08006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.320+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc1004b170 con 0x7fbc201081f0 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.497+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fbc201031a0 con 0x7fbc0c038f70 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.497+0000 7fbc17fff640 1 -- 172.21.15.3:0/57300529 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fbc201031a0 con 0x7fbc0c038f70 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.499+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 msgr2=0x7fbc0c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.499+0000 7fbc25390640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 0x7fbc0c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbc080098c0 tx=0x7fbc08006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.499+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 msgr2=0x7fbc2019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.499+0000 7fbc25390640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbc100097f0 tx=0x7fbc10030ec0 comp rx=0 tx=0).stop 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 shutdown_connections 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fbc0c038f70 0x7fbc0c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:16.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 --2- 172.21.15.3:0/57300529 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc201081f0 0x7fbc2019b7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:16.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 >> 172.21.15.3:0/57300529 conn(0x7fbc20101e10 msgr2=0x7fbc20102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:16.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 shutdown_connections 2024-09-12T02:09:16.050 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:15.500+0000 7fbc25390640 1 -- 172.21.15.3:0/57300529 wait complete. 2024-09-12T02:09:16.050 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-09-12T02:09:17.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:16 smithi003 ceph-mon[31476]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:17.407 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: Generating public/private rsa key pair. 2024-09-12T02:09:17.407 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: Your identification has been saved in /tmp/tmpvsiu604i/key 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: Your public key has been saved in /tmp/tmpvsiu604i/key.pub 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: The key fingerprint is: 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: SHA256:1jfCwzXQ18DpkFK7y2YwvwJ9l8rB4/oTW+6AtfwgIVQ ceph-c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: The key's randomart image is: 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: +---[RSA 3072]----+ 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | Eo.o.+ | 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | ...+.+ .| 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | . .++ | 2024-09-12T02:09:17.408 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | . + . o. | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | S.X.= . | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | ....&*++ | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | .o+@X | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | .+*=. | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: | .+ooo | 2024-09-12T02:09:17.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: +----[SHA256]-----+ 2024-09-12T02:09:17.673 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.491+0000 7fc106f29640 1 Processor -- start 2024-09-12T02:09:17.673 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.491+0000 7fc106f29640 1 -- start start 2024-09-12T02:09:17.673 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.491+0000 7fc106f29640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:17.673 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.491+0000 7fc106f29640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc100108150 con 0x7fc100107780 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.492+0000 7fc104cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.492+0000 7fc104cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48898/0 (socket says 172.21.15.3:48898) 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.492+0000 7fc104cc5640 1 -- 172.21.15.3:0/2026637015 learned_addr learned my addr 172.21.15.3:0/2026637015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.492+0000 7fc104cc5640 1 -- 172.21.15.3:0/2026637015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc100108930 con 0x7fc100107780 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc104cc5640 1 --2- 172.21.15.3:0/2026637015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e80 tx=0x7fc0f802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2da8694262729e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2026637015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0f8036070 con 0x7fc100107780 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2026637015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc0f80318c0 con 0x7fc100107780 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc0ff7fe640 1 -- 172.21.15.3:0/2026637015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0f8007ab0 con 0x7fc100107780 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc106f29640 1 -- 172.21.15.3:0/2026637015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 msgr2=0x7fc100107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.493+0000 7fc106f29640 1 --2- 172.21.15.3:0/2026637015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009e80 tx=0x7fc0f802a7a0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- 172.21.15.3:0/2026637015 shutdown_connections 2024-09-12T02:09:17.674 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 --2- 172.21.15.3:0/2026637015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc100107b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- 172.21.15.3:0/2026637015 >> 172.21.15.3:0/2026637015 conn(0x7fc100102f30 msgr2=0x7fc100105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- 172.21.15.3:0/2026637015 shutdown_connections 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- 172.21.15.3:0/2026637015 wait complete. 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 Processor -- start 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- start start 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.494+0000 7fc106f29640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc10010bce0 con 0x7fc100107780 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc104cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc104cc5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48912/0 (socket says 172.21.15.3:48912) 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc104cc5640 1 -- 172.21.15.3:0/2963091171 learned_addr learned my addr 172.21.15.3:0/2963091171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc104cc5640 1 -- 172.21.15.3:0/2963091171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10019def0 con 0x7fc100107780 2024-09-12T02:09:17.675 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc104cc5640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc0f802acc0 tx=0x7fc0f8031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0f8036050 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc0f8037040 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0f803d8c0 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.495+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10019e180 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.496+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001a0e70 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.496+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc0f8044020 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.497+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10010be60 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.497+0000 7fc0fdffb640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 0x7fc0ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.497+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc0f8042020 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.497+0000 7fc0fffff640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 0x7fc0ec03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.497+0000 7fc0fffff640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 0x7fc0ec03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80098c0 tx=0x7fc0e8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.502+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0f804b020 con 0x7fc100107780 2024-09-12T02:09:17.676 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:16.681+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fc100104160 con 0x7fc0ec038fc0 2024-09-12T02:09:17.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.124+0000 7fc0fdffb640 1 -- 172.21.15.3:0/2963091171 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc100104160 con 0x7fc0ec038fc0 2024-09-12T02:09:17.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 msgr2=0x7fc0ec03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:17.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 0x7fc0ec03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80098c0 tx=0x7fc0e8006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 msgr2=0x7fc10019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:17.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc0f802acc0 tx=0x7fc0f8031ca0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 shutdown_connections 2024-09-12T02:09:17.680 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fc0ec038fc0 0x7fc0ec03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 --2- 172.21.15.3:0/2963091171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc100107780 0x7fc10019d9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:17.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 >> 172.21.15.3:0/2963091171 conn(0x7fc100102f30 msgr2=0x7fc100103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:17.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 shutdown_connections 2024-09-12T02:09:17.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:17.127+0000 7fc106f29640 1 -- 172.21.15.3:0/2963091171 wait complete. 2024-09-12T02:09:17.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-mon[31476]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:17.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-mon[31476]: Generating ssh key... 2024-09-12T02:09:17.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:17.968 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:17 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.048+0000 7fde6673f640 1 Processor -- start 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde6673f640 1 -- start start 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde6673f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde6673f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde60108100 con 0x7fde60107730 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:18.746 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48928/0 (socket says 172.21.15.3:48928) 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde5ffff640 1 -- 172.21.15.3:0/1134359148 learned_addr learned my addr 172.21.15.3:0/1134359148 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.049+0000 7fde5ffff640 1 -- 172.21.15.3:0/1134359148 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde60108930 con 0x7fde60107730 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.050+0000 7fde5ffff640 1 --2- 172.21.15.3:0/1134359148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde50009820 tx=0x7fde5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f0beec1f4c4820 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.050+0000 7fde5effd640 1 -- 172.21.15.3:0/1134359148 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde50037070 con 0x7fde60107730 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.050+0000 7fde5effd640 1 -- 172.21.15.3:0/1134359148 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde50032900 con 0x7fde60107730 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.050+0000 7fde5effd640 1 -- 172.21.15.3:0/1134359148 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde50030850 con 0x7fde60107730 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- 172.21.15.3:0/1134359148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 msgr2=0x7fde60107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 --2- 172.21.15.3:0/1134359148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde50009820 tx=0x7fde5002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- 172.21.15.3:0/1134359148 shutdown_connections 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 --2- 172.21.15.3:0/1134359148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde60107b30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- 172.21.15.3:0/1134359148 >> 172.21.15.3:0/1134359148 conn(0x7fde60102fa0 msgr2=0x7fde601053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:18.747 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- 172.21.15.3:0/1134359148 shutdown_connections 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- 172.21.15.3:0/1134359148 wait complete. 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 Processor -- start 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.051+0000 7fde6673f640 1 -- start start 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde6673f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde6673f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010bce0 con 0x7fde60107730 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48942/0 (socket says 172.21.15.3:48942) 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5ffff640 1 -- 172.21.15.3:0/4242979836 learned_addr learned my addr 172.21.15.3:0/4242979836 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5ffff640 1 -- 172.21.15.3:0/4242979836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6019bc90 con 0x7fde60107730 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5ffff640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde500097f0 tx=0x7fde50030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.052+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde50037040 con 0x7fde60107730 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.053+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde50040070 con 0x7fde60107730 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.053+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde5003a390 con 0x7fde60107730 2024-09-12T02:09:18.748 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.053+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6019bf20 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.053+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6019ec10 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.053+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fde5003a5e0 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.054+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde60107bb0 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.054+0000 7fde5d7fa640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 0x7fde4803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.054+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde5006c560 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.054+0000 7fde5f7fe640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 0x7fde4803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.054+0000 7fde5f7fe640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 0x7fde4803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fde540098c0 tx=0x7fde54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.058+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde50043760 con 0x7fde60107730 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.236+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fde60104220 con 0x7fde48038fc0 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.237+0000 7fde5d7fa640 1 -- 172.21.15.3:0/4242979836 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fde60104220 con 0x7fde48038fc0 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 msgr2=0x7fde4803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:18.749 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 0x7fde4803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fde540098c0 tx=0x7fde54006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 msgr2=0x7fde6019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde500097f0 tx=0x7fde50030d80 comp rx=0 tx=0).stop 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 shutdown_connections 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fde48038fc0 0x7fde4803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 --2- 172.21.15.3:0/4242979836 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fde60107730 0x7fde6019b750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 >> 172.21.15.3:0/4242979836 conn(0x7fde60102fa0 msgr2=0x7fde60103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 shutdown_connections 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:18.239+0000 7fde6673f640 1 -- 172.21.15.3:0/4242979836 wait complete. 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:09:18.750 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T02:09:18.751 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-09-12T02:09:19.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:19 smithi003 ceph-mon[31476]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:20.359 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:20 smithi003 ceph-mon[31476]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:21.262 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:21 smithi003 ceph-mon[31476]: Deploying cephadm binary to smithi003 2024-09-12T02:09:23.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-09-12T02:09:23.779 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5cdcb640 1 Processor -- start 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5cdcb640 1 -- start start 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5cdcb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5cdcb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c58108130 con 0x7f2c58107760 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.150+0000 7f2c5659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36354/0 (socket says 172.21.15.3:36354) 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.151+0000 7f2c5659c640 1 -- 172.21.15.3:0/1720833721 learned_addr learned my addr 172.21.15.3:0/1720833721 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.151+0000 7f2c5659c640 1 -- 172.21.15.3:0/1720833721 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58108960 con 0x7f2c58107760 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c5659c640 1 --2- 172.21.15.3:0/1720833721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009820 tx=0x7f2c4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b8be7e0ee2db9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c55d9b640 1 -- 172.21.15.3:0/1720833721 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48037070 con 0x7f2c58107760 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c55d9b640 1 -- 172.21.15.3:0/1720833721 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c48032900 con 0x7f2c58107760 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c55d9b640 1 -- 172.21.15.3:0/1720833721 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48030850 con 0x7f2c58107760 2024-09-12T02:09:23.780 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/1720833721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 msgr2=0x7f2c58107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/1720833721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009820 tx=0x7f2c4802a270 comp rx=0 tx=0).stop 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/1720833721 shutdown_connections 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.152+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/1720833721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c58107b60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.153+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/1720833721 >> 172.21.15.3:0/1720833721 conn(0x7f2c58102ff0 msgr2=0x7f2c58105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.153+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/1720833721 shutdown_connections 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.153+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/1720833721 wait complete. 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.153+0000 7f2c5cdcb640 1 Processor -- start 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.153+0000 7f2c5cdcb640 1 -- start start 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5cdcb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5cdcb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5810bd10 con 0x7f2c58107760 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36360/0 (socket says 172.21.15.3:36360) 2024-09-12T02:09:23.781 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5659c640 1 -- 172.21.15.3:0/3627255097 learned_addr learned my addr 172.21.15.3:0/3627255097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5659c640 1 -- 172.21.15.3:0/3627255097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5819bcd0 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5659c640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009940 tx=0x7f2c48032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48042070 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c480308c0 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5819bf60 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.154+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c48037040 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.155+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5819ec50 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.155+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2c48045070 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.156+0000 7f2c4f7fe640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 0x7f2c4403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.156+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c4806c060 con 0x7f2c58107760 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.156+0000 7f2c4ffff640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 0x7f2c4403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:23.782 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.156+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5810be90 con 0x7f2c58107760 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.156+0000 7f2c4ffff640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 0x7f2c4403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c400098c0 tx=0x7f2c40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.161+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c48045520 con 0x7f2c58107760 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:19.339+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}) -- 0x7f2c581042e0 con 0x7f2c44038fc0 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.209+0000 7f2c4f7fe640 1 -- 172.21.15.3:0/3627255097 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f2c581042e0 con 0x7f2c44038fc0 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 msgr2=0x7f2c4403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 0x7f2c4403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2c400098c0 tx=0x7f2c40006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 msgr2=0x7f2c5819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c48009940 tx=0x7f2c48032ce0 comp rx=0 tx=0).stop 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 shutdown_connections 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f2c44038fc0 0x7f2c4403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 --2- 172.21.15.3:0/3627255097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2c58107760 0x7f2c5819b790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 >> 172.21.15.3:0/3627255097 conn(0x7f2c58102ff0 msgr2=0x7f2c58103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:23.783 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 shutdown_connections 2024-09-12T02:09:23.784 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:23.212+0000 7f2c5cdcb640 1 -- 172.21.15.3:0/3627255097 wait complete. 2024-09-12T02:09:23.784 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-09-12T02:09:24.437 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:24 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:24.437 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:24 smithi003 ceph-mon[31476]: Added host smithi003 2024-09-12T02:09:24.437 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:24 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f97240ad640 1 Processor -- start 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f97240ad640 1 -- start start 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f97240ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f97240ad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f971c07bdf0 con 0x7f971c106040 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f9721e49640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f9721e49640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36362/0 (socket says 172.21.15.3:36362) 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.268+0000 7f9721e49640 1 -- 172.21.15.3:0/3093318754 learned_addr learned my addr 172.21.15.3:0/3093318754 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.269+0000 7f9721e49640 1 -- 172.21.15.3:0/3093318754 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c106980 con 0x7f971c106040 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.269+0000 7f9721e49640 1 --2- 172.21.15.3:0/3093318754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9718009820 tx=0x7f971802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c910ad702795d11b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:25.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.269+0000 7f9720e47640 1 -- 172.21.15.3:0/3093318754 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9718037070 con 0x7f971c106040 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f9720e47640 1 -- 172.21.15.3:0/3093318754 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9718032900 con 0x7f971c106040 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f9720e47640 1 -- 172.21.15.3:0/3093318754 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9718030850 con 0x7f971c106040 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 -- 172.21.15.3:0/3093318754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 msgr2=0x7f971c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 --2- 172.21.15.3:0/3093318754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9718009820 tx=0x7f971802a270 comp rx=0 tx=0).stop 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 -- 172.21.15.3:0/3093318754 shutdown_connections 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 --2- 172.21.15.3:0/3093318754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c106440 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 -- 172.21.15.3:0/3093318754 >> 172.21.15.3:0/3093318754 conn(0x7f971c101e50 msgr2=0x7f971c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 -- 172.21.15.3:0/3093318754 shutdown_connections 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.270+0000 7f97240ad640 1 -- 172.21.15.3:0/3093318754 wait complete. 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f97240ad640 1 Processor -- start 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f97240ad640 1 -- start start 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f97240ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:25.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f97240ad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f971c107a80 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f9721e49640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f9721e49640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36368/0 (socket says 172.21.15.3:36368) 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f9721e49640 1 -- 172.21.15.3:0/2602869383 learned_addr learned my addr 172.21.15.3:0/2602869383 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f9721e49640 1 -- 172.21.15.3:0/2602869383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c19de70 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.271+0000 7f9721e49640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f97180097f0 tx=0x7f9718030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9718037040 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c19e100 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9718040070 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f971803a390 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1a0df0 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.272+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f971803a630 con 0x7f971c106040 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.273+0000 7f9712ffd640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 0x7f970803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:25.234 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.273+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f971c106440 con 0x7f971c106040 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.273+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f971806c4b0 con 0x7f971c106040 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.273+0000 7f9721648640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 0x7f970803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.273+0000 7f9721648640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 0x7f970803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098c0 tx=0x7f970c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.278+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9718042310 con 0x7f971c106040 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.468+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f971c062630 con 0x7f9708038f70 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.473+0000 7f9712ffd640 1 -- 172.21.15.3:0/2602869383 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f971c062630 con 0x7f9708038f70 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 msgr2=0x7f970803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 0x7f970803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f970c0098c0 tx=0x7f970c006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:25.235 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 msgr2=0x7f971c19d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f97180097f0 tx=0x7f9718030d80 comp rx=0 tx=0).stop 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 shutdown_connections 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f9708038f70 0x7f970803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 --2- 172.21.15.3:0/2602869383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f971c106040 0x7f971c19d930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 >> 172.21.15.3:0/2602869383 conn(0x7f971c101e50 msgr2=0x7f971c191c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 shutdown_connections 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:24.476+0000 7f97240ad640 1 -- 172.21.15.3:0/2602869383 wait complete. 2024-09-12T02:09:25.236 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-09-12T02:09:25.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:25 smithi003 ceph-mon[31476]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:25 smithi003 ceph-mon[31476]: Saving service mon spec with placement count:5 2024-09-12T02:09:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:25 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:26.665 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.647+0000 7f339fbbe640 1 Processor -- start 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.647+0000 7f339fbbe640 1 -- start start 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339fbbe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339fbbe640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f339807be50 con 0x7f3398108170 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339d95a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339d95a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36376/0 (socket says 172.21.15.3:36376) 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339d95a640 1 -- 172.21.15.3:0/627589063 learned_addr learned my addr 172.21.15.3:0/627589063 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.648+0000 7f339d95a640 1 -- 172.21.15.3:0/627589063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339810aaa0 con 0x7f3398108170 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.649+0000 7f339d95a640 1 --2- 172.21.15.3:0/627589063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3388009e80 tx=0x7f338802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0cd7d6a22a7e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.649+0000 7f339c958640 1 -- 172.21.15.3:0/627589063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388036070 con 0x7f3398108170 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.649+0000 7f339c958640 1 -- 172.21.15.3:0/627589063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33880318c0 con 0x7f3398108170 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.649+0000 7f339c958640 1 -- 172.21.15.3:0/627589063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388007ab0 con 0x7f3398108170 2024-09-12T02:09:26.666 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 -- 172.21.15.3:0/627589063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 msgr2=0x7f339810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 --2- 172.21.15.3:0/627589063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3388009e80 tx=0x7f338802a7a0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 -- 172.21.15.3:0/627589063 shutdown_connections 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 --2- 172.21.15.3:0/627589063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f339810a560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 -- 172.21.15.3:0/627589063 >> 172.21.15.3:0/627589063 conn(0x7f3398101e10 msgr2=0x7f3398104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 -- 172.21.15.3:0/627589063 shutdown_connections 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.650+0000 7f339fbbe640 1 -- 172.21.15.3:0/627589063 wait complete. 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339fbbe640 1 Processor -- start 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339fbbe640 1 -- start start 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339fbbe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339fbbe640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f339810bb40 con 0x7f3398108170 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339d95a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.667 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339d95a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36384/0 (socket says 172.21.15.3:36384) 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339d95a640 1 -- 172.21.15.3:0/3383831860 learned_addr learned my addr 172.21.15.3:0/3383831860 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339d95a640 1 -- 172.21.15.3:0/3383831860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3398107f30 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.651+0000 7f339d95a640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f338802acc0 tx=0x7f3388031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.652+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3388036050 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.652+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3388037040 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.652+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3398106120 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.652+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f338803d920 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.652+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3398106660 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3388044020 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f3396ffd640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 0x7f338003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3388042020 con 0x7f3398108170 2024-09-12T02:09:26.668 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f339d159640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 0x7f338003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33981085d0 con 0x7f3398108170 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.653+0000 7f339d159640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 0x7f338003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f338c0098c0 tx=0x7f338c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.658+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f338804b020 con 0x7f3398108170 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.848+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3398106dc0 con 0x7f3380038fc0 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.853+0000 7f3396ffd640 1 -- 172.21.15.3:0/3383831860 <== mgr.14118 v2:172.21.15.3:6800/1539065682 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3398106dc0 con 0x7f3380038fc0 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 msgr2=0x7f338003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 0x7f338003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f338c0098c0 tx=0x7f338c006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 msgr2=0x7f33981079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f338802acc0 tx=0x7f3388031ca0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 shutdown_connections 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f3380038fc0 0x7f338003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 --2- 172.21.15.3:0/3383831860 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3398108170 0x7f33981079f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:26.669 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 >> 172.21.15.3:0/3383831860 conn(0x7f3398101e10 msgr2=0x7f3398102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:26.670 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 shutdown_connections 2024-09-12T02:09:26.670 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:25.857+0000 7f339fbbe640 1 -- 172.21.15.3:0/3383831860 wait complete. 2024-09-12T02:09:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:26 smithi003 ceph-mon[31476]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:26 smithi003 ceph-mon[31476]: Saving service mgr spec with placement count:2 2024-09-12T02:09:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:26 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:28.333 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:28 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:28.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:28 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:28.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:28 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4279785927' entity='client.admin' 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.590+0000 7fb01bbd6640 1 Processor -- start 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.590+0000 7fb01bbd6640 1 -- start start 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.590+0000 7fb01bbd6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.590+0000 7fb01bbd6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb014106fd0 con 0x7fb014106600 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.591+0000 7fb019972640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.591+0000 7fb019972640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36390/0 (socket says 172.21.15.3:36390) 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.591+0000 7fb019972640 1 -- 172.21.15.3:0/708419548 learned_addr learned my addr 172.21.15.3:0/708419548 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.591+0000 7fb019972640 1 -- 172.21.15.3:0/708419548 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb014107800 con 0x7fb014106600 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.592+0000 7fb019972640 1 --2- 172.21.15.3:0/708419548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb010009820 tx=0x7fb01002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2604814b4d4b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:29.037 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.592+0000 7fb018970640 1 -- 172.21.15.3:0/708419548 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb010037070 con 0x7fb014106600 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.592+0000 7fb018970640 1 -- 172.21.15.3:0/708419548 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb010032920 con 0x7fb014106600 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.592+0000 7fb018970640 1 -- 172.21.15.3:0/708419548 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb010030810 con 0x7fb014106600 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 -- 172.21.15.3:0/708419548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 msgr2=0x7fb014106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/708419548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb010009820 tx=0x7fb01002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 -- 172.21.15.3:0/708419548 shutdown_connections 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/708419548 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb014106a00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 -- 172.21.15.3:0/708419548 >> 172.21.15.3:0/708419548 conn(0x7fb014101e10 msgr2=0x7fb014104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 -- 172.21.15.3:0/708419548 shutdown_connections 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 -- 172.21.15.3:0/708419548 wait complete. 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.593+0000 7fb01bbd6640 1 Processor -- start 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb01bbd6640 1 -- start start 2024-09-12T02:09:29.038 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb01bbd6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb01bbd6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb01410abb0 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb019972640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb019972640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36402/0 (socket says 172.21.15.3:36402) 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb019972640 1 -- 172.21.15.3:0/4279785927 learned_addr learned my addr 172.21.15.3:0/4279785927 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb019972640 1 -- 172.21.15.3:0/4279785927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01419df60 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb019972640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb010030560 tx=0x7fb010032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.594+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb010037040 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.595+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01419e1f0 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.595+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb010040070 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.595+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0141a0ee0 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.595+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb01003a6d0 con 0x7fb014106600 2024-09-12T02:09:29.039 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.595+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb01003a870 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.596+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01410ad30 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.596+0000 7fb00affd640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 0x7fb00003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.596+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb01006c4e0 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.596+0000 7fb019171640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 0x7fb00003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.596+0000 7fb019171640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 0x7fb00003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb0040098c0 tx=0x7fb004006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.601+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb010041270 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.797+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb014106a80 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.799+0000 7fb00affd640 1 -- 172.21.15.3:0/4279785927 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fb010045460 con 0x7fb014106600 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 msgr2=0x7fb00003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 0x7fb00003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb0040098c0 tx=0x7fb004006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 msgr2=0x7fb01419da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:29.040 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb010030560 tx=0x7fb010032670 comp rx=0 tx=0).stop 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 shutdown_connections 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7fb000038fc0 0x7fb00003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 --2- 172.21.15.3:0/4279785927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb014106600 0x7fb01419da20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 >> 172.21.15.3:0/4279785927 conn(0x7fb014101e10 msgr2=0x7fb014102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 shutdown_connections 2024-09-12T02:09:29.041 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:27.802+0000 7fb01bbd6640 1 -- 172.21.15.3:0/4279785927 wait complete. 2024-09-12T02:09:29.986 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:29 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:29.986 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:29 smithi003 ceph-mon[31476]: from='mgr.14118 172.21.15.3:0/1720640992' entity='mgr.x' 2024-09-12T02:09:29.986 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:29 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1483602363' entity='client.admin' 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.480+0000 7f5984b63640 1 Processor -- start 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.480+0000 7f5984b63640 1 -- start start 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f5984b63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f5984b63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5980108190 con 0x7f59801077c0 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f597e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f597e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51834/0 (socket says 172.21.15.3:51834) 2024-09-12T02:09:30.243 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f597e59c640 1 -- 172.21.15.3:0/1296922579 learned_addr learned my addr 172.21.15.3:0/1296922579 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.481+0000 7f597e59c640 1 -- 172.21.15.3:0/1296922579 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5980108920 con 0x7f59801077c0 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.482+0000 7f597e59c640 1 --2- 172.21.15.3:0/1296922579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5970009820 tx=0x7f597002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2653b562ea04c23b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.482+0000 7f597d59a640 1 -- 172.21.15.3:0/1296922579 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5970037070 con 0x7f59801077c0 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.482+0000 7f597d59a640 1 -- 172.21.15.3:0/1296922579 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5970032920 con 0x7f59801077c0 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.482+0000 7f597d59a640 1 -- 172.21.15.3:0/1296922579 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5970030810 con 0x7f59801077c0 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 -- 172.21.15.3:0/1296922579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 msgr2=0x7f5980107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 --2- 172.21.15.3:0/1296922579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5970009820 tx=0x7f597002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:30.244 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 -- 172.21.15.3:0/1296922579 shutdown_connections 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 --2- 172.21.15.3:0/1296922579 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f5980107bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 -- 172.21.15.3:0/1296922579 >> 172.21.15.3:0/1296922579 conn(0x7f5980102fb0 msgr2=0x7f59801053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 -- 172.21.15.3:0/1296922579 shutdown_connections 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.483+0000 7f5984b63640 1 -- 172.21.15.3:0/1296922579 wait complete. 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.484+0000 7f5984b63640 1 Processor -- start 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.484+0000 7f5984b63640 1 -- start start 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f5984b63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f5984b63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f598010bcd0 con 0x7f59801077c0 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f597e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f597e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51840/0 (socket says 172.21.15.3:51840) 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f597e59c640 1 -- 172.21.15.3:0/1483602363 learned_addr learned my addr 172.21.15.3:0/1483602363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f597e59c640 1 -- 172.21.15.3:0/1483602363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598007e050 con 0x7f59801077c0 2024-09-12T02:09:30.245 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.485+0000 7f597e59c640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59700097f0 tx=0x7f59700327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.486+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5970037040 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.486+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5970030cc0 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.486+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598007c240 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.486+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f597003a640 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.486+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598007c6c0 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.487+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f598010be50 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.487+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f597003a940 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.488+0000 7f596f7fe640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 0x7f596803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.488+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f597006c550 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.488+0000 7f597dd9b640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 0x7f596803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.488+0000 7f597dd9b640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 0x7f596803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59740098c0 tx=0x7f5974006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.493+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f597006bee0 con 0x7f59801077c0 2024-09-12T02:09:30.246 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.675+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f598007ca70 con 0x7f59801077c0 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.679+0000 7f596f7fe640 1 -- 172.21.15.3:0/1483602363 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f598007ca70 con 0x7f59801077c0 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 msgr2=0x7f596803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 0x7f596803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59740098c0 tx=0x7f5974006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 msgr2=0x7f598007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59700097f0 tx=0x7f59700327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 shutdown_connections 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f5968038f70 0x7f596803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.681+0000 7f5984b63640 1 --2- 172.21.15.3:0/1483602363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59801077c0 0x7f598007db10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.682+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 >> 172.21.15.3:0/1483602363 conn(0x7f5980102fb0 msgr2=0x7f5980103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.682+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 shutdown_connections 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:29.682+0000 7f5984b63640 1 -- 172.21.15.3:0/1483602363 wait complete. 2024-09-12T02:09:30.247 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-09-12T02:09:31.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1725268210' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T02:09:31.474 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ignoring --setuser ceph since I am not root 2024-09-12T02:09:31.474 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ignoring --setgroup ceph since I am not root 2024-09-12T02:09:31.474 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:31.414+0000 7fb67ccfc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:09:31.474 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:31.473+0000 7fb67ccfc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:09:31.777 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:31.630+0000 7fb67ccfc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:09:31.778 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:31.699+0000 7fb67ccfc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:09:32.199 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:31.857+0000 7fb67ccfc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:09:32.199 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:32.005+0000 7fb67ccfc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.973+0000 7f5a19edf640 1 Processor -- start 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a19edf640 1 -- start start 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a19edf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a19edf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14106fd0 con 0x7f5a14106600 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51848/0 (socket says 172.21.15.3:51848) 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a137fe640 1 -- 172.21.15.3:0/4001468661 learned_addr learned my addr 172.21.15.3:0/4001468661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.974+0000 7f5a137fe640 1 -- 172.21.15.3:0/4001468661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a14107800 con 0x7f5a14106600 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.975+0000 7f5a137fe640 1 --2- 172.21.15.3:0/4001468661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009820 tx=0x7f5a0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705c37a32b22399d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.975+0000 7f5a127fc640 1 -- 172.21.15.3:0/4001468661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a04037070 con 0x7f5a14106600 2024-09-12T02:09:32.207 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.975+0000 7f5a127fc640 1 -- 172.21.15.3:0/4001468661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a04032920 con 0x7f5a14106600 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.975+0000 7f5a127fc640 1 -- 172.21.15.3:0/4001468661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a04030810 con 0x7f5a14106600 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 -- 172.21.15.3:0/4001468661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 msgr2=0x7f5a14106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 --2- 172.21.15.3:0/4001468661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009820 tx=0x7f5a0402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 -- 172.21.15.3:0/4001468661 shutdown_connections 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 --2- 172.21.15.3:0/4001468661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14106a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 -- 172.21.15.3:0/4001468661 >> 172.21.15.3:0/4001468661 conn(0x7f5a14101e10 msgr2=0x7f5a14104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 -- 172.21.15.3:0/4001468661 shutdown_connections 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.976+0000 7f5a19edf640 1 -- 172.21.15.3:0/4001468661 wait complete. 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a19edf640 1 Processor -- start 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a19edf640 1 -- start start 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a19edf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a19edf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1410abb0 con 0x7f5a14106600 2024-09-12T02:09:32.208 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51864/0 (socket says 172.21.15.3:51864) 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a137fe640 1 -- 172.21.15.3:0/1725268210 learned_addr learned my addr 172.21.15.3:0/1725268210 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.977+0000 7f5a137fe640 1 -- 172.21.15.3:0/1725268210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a14191d70 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a137fe640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a040097f0 tx=0x7f5a040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a04037040 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a04030c40 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a0403a5b0 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1418ff60 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.978+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141904a0 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5a0403a7a0 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1410ad30 con 0x7f5a14106600 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a10ff9640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 0x7f59fc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:32.209 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a0406c390 con 0x7f5a14106600 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a12ffd640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 0x7f59fc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.979+0000 7f5a12ffd640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 0x7f59fc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a080098c0 tx=0x7f5a08006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:30.984+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a040371e0 con 0x7f5a14106600 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.209+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5a14195560 con 0x7f5a14106600 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.261+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5a04068f20 con 0x7f5a14106600 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.263+0000 7f5a10ff9640 1 -- 172.21.15.3:0/1725268210 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5a0406ae40 con 0x7f5a14106600 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 msgr2=0x7f59fc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 0x7f59fc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a080098c0 tx=0x7f5a08006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 msgr2=0x7f5a14191830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a040097f0 tx=0x7f5a040327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 shutdown_connections 2024-09-12T02:09:32.210 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f59fc038fc0 0x7f59fc03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.211 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 --2- 172.21.15.3:0/1725268210 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a14106600 0x7f5a14191830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:32.211 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 >> 172.21.15.3:0/1725268210 conn(0x7f5a14101e10 msgr2=0x7f5a14102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:32.211 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 shutdown_connections 2024-09-12T02:09:32.211 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:31.264+0000 7f5a19edf640 1 -- 172.21.15.3:0/1725268210 wait complete. 2024-09-12T02:09:32.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1725268210' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T02:09:32.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-mon[31476]: mgrmap e9: x(active, since 19s) 2024-09-12T02:09:32.477 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:32.337+0000 7fb67ccfc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:09:32.477 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:32.401+0000 7fb67ccfc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:09:32.777 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:32.597+0000 7fb67ccfc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:09:33.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:33 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1329707759' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:09:33.505 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.701+0000 7f82bc20d640 1 Processor -- start 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.701+0000 7f82bc20d640 1 -- start start 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.701+0000 7f82bc20d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.701+0000 7f82bc20d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41081d0 con 0x7f82b4107800 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.702+0000 7f82b9fa9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.702+0000 7f82b9fa9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51886/0 (socket says 172.21.15.3:51886) 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.702+0000 7f82b9fa9640 1 -- 172.21.15.3:0/2987483163 learned_addr learned my addr 172.21.15.3:0/2987483163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.702+0000 7f82b9fa9640 1 -- 172.21.15.3:0/2987483163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41089b0 con 0x7f82b4107800 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82b9fa9640 1 --2- 172.21.15.3:0/2987483163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009820 tx=0x7f82b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4508059a0f30b9ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82b8fa7640 1 -- 172.21.15.3:0/2987483163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b0037070 con 0x7f82b4107800 2024-09-12T02:09:33.506 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82b8fa7640 1 -- 172.21.15.3:0/2987483163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b0032920 con 0x7f82b4107800 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82b8fa7640 1 -- 172.21.15.3:0/2987483163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b0030810 con 0x7f82b4107800 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82bc20d640 1 -- 172.21.15.3:0/2987483163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 msgr2=0x7f82b4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82bc20d640 1 --2- 172.21.15.3:0/2987483163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009820 tx=0x7f82b002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82bc20d640 1 -- 172.21.15.3:0/2987483163 shutdown_connections 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82bc20d640 1 --2- 172.21.15.3:0/2987483163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b4107c00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.703+0000 7f82bc20d640 1 -- 172.21.15.3:0/2987483163 >> 172.21.15.3:0/2987483163 conn(0x7f82b4102ff0 msgr2=0x7f82b4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 -- 172.21.15.3:0/2987483163 shutdown_connections 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 -- 172.21.15.3:0/2987483163 wait complete. 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 Processor -- start 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 -- start start 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82bc20d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82b410bd60 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.704+0000 7f82b9fa9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82b9fa9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51898/0 (socket says 172.21.15.3:51898) 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82b9fa9640 1 -- 172.21.15.3:0/1329707759 learned_addr learned my addr 172.21.15.3:0/1329707759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82b9fa9640 1 -- 172.21.15.3:0/1329707759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b419df50 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82b9fa9640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b0030560 tx=0x7f82b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b0037040 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b0040070 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b419e1e0 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b003a740 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.705+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a0ed0 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f82b003b420 con 0x7f82b4107800 2024-09-12T02:09:33.508 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b410bee0 con 0x7f82b4107800 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82aaffd640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 0x7f82a003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82b97a8640 1 -- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 msgr2=0x7f82a003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82b006c5e0 con 0x7f82b4107800 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.706+0000 7f82b97a8640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 0x7f82a003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.711+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82b004a200 con 0x7f82b4107800 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.907+0000 7f82b97a8640 1 -- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 msgr2=0x7f82a003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.907+0000 7f82b97a8640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 0x7f82a003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.936+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f82b41a10a0 con 0x7f82b4107800 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.936+0000 7f82aaffd640 1 -- 172.21.15.3:0/1329707759 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f82b0031e80 con 0x7f82b4107800 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 msgr2=0x7f82a003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:09:33.509 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 0x7f82a003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 msgr2=0x7f82b419da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82b0030560 tx=0x7f82b0032670 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 shutdown_connections 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f82a0039020 0x7f82a003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 --2- 172.21.15.3:0/1329707759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82b4107800 0x7f82b419da10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.938+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 >> 172.21.15.3:0/1329707759 conn(0x7f82b4102ff0 msgr2=0x7f82b4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.939+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 shutdown_connections 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:32.939+0000 7f82bc20d640 1 -- 172.21.15.3:0/1329707759 wait complete. 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-09-12T02:09:33.510 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2024-09-12T02:09:33.747 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:33 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:33.544+0000 7fb67ccfc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:09:33.748 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:33 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:33.615+0000 7fb67ccfc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:09:34.496 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.375+0000 7fb67ccfc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:09:34.496 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.496+0000 7fb67ccfc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:09:34.778 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.646+0000 7fb67ccfc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:09:34.778 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.711+0000 7fb67ccfc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:09:35.168 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.868+0000 7fb67ccfc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:09:35.168 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:34.939+0000 7fb67ccfc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:09:35.168 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.001+0000 7fb67ccfc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:09:35.168 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.167+0000 7fb67ccfc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:09:35.454 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.235+0000 7fb67ccfc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:09:35.454 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.305+0000 7fb67ccfc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:09:35.454 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.392+0000 7fb67ccfc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:09:35.454 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.454+0000 7fb67ccfc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:09:35.777 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:35.516+0000 7fb67ccfc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:36.345+0000 7fb67ccfc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: Active manager daemon x restarted 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: Activating manager daemon x 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: mgrmap e10: x(active, starting, since 0.00356077s) 2024-09-12T02:09:36.602 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: Manager daemon x is now available 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:09:36.603 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f9908aed640 1 Processor -- start 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f9908aed640 1 -- start start 2024-09-12T02:09:38.263 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f9908aed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f9908aed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f990407be10 con 0x7f99041081b0 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f990259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f990259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51902/0 (socket says 172.21.15.3:51902) 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.924+0000 7f990259c640 1 -- 172.21.15.3:0/2104737424 learned_addr learned my addr 172.21.15.3:0/2104737424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.925+0000 7f990259c640 1 -- 172.21.15.3:0/2104737424 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990410aae0 con 0x7f99041081b0 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.925+0000 7f990259c640 1 --2- 172.21.15.3:0/2104737424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009820 tx=0x7f98f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9671dc7c40bfd75a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:38.264 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9901d9b640 1 -- 172.21.15.3:0/2104737424 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4037070 con 0x7f99041081b0 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9901d9b640 1 -- 172.21.15.3:0/2104737424 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f4032920 con 0x7f99041081b0 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9901d9b640 1 -- 172.21.15.3:0/2104737424 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4030810 con 0x7f99041081b0 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9908aed640 1 -- 172.21.15.3:0/2104737424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 msgr2=0x7f990410a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9908aed640 1 --2- 172.21.15.3:0/2104737424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009820 tx=0x7f98f402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9908aed640 1 -- 172.21.15.3:0/2104737424 shutdown_connections 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9908aed640 1 --2- 172.21.15.3:0/2104737424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990410a5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.926+0000 7f9908aed640 1 -- 172.21.15.3:0/2104737424 >> 172.21.15.3:0/2104737424 conn(0x7f9904101e10 msgr2=0x7f9904104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.927+0000 7f9908aed640 1 -- 172.21.15.3:0/2104737424 shutdown_connections 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.927+0000 7f9908aed640 1 -- 172.21.15.3:0/2104737424 wait complete. 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.927+0000 7f9908aed640 1 Processor -- start 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.927+0000 7f9908aed640 1 -- start start 2024-09-12T02:09:38.265 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f9908aed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f9908aed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f990410bb60 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f990259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f990259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51908/0 (socket says 172.21.15.3:51908) 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f990259c640 1 -- 172.21.15.3:0/1806916451 learned_addr learned my addr 172.21.15.3:0/1806916451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f990259c640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990419bcc0 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f990259c640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097f0 tx=0x7f98f40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4037040 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990419bf50 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f4030c40 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f403a5d0 con 0x7f99041081b0 2024-09-12T02:09:38.266 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.928+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990419ec40 con 0x7f99041081b0 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.929+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f98f403b3f0 con 0x7f99041081b0 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.930+0000 7f98fb7fe640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.930+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f98f003bb90 con 0x7f98f0038fc0 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.930+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f98f406c4b0 con 0x7f99041081b0 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.930+0000 7f98fbfff640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 msgr2=0x7f98f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:33.930+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:09:38.267 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:34.131+0000 7f98fbfff640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 msgr2=0x7f98f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:34.131+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:34.532+0000 7f98fbfff640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 msgr2=0x7f98f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:34.532+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:35.333+0000 7f98fbfff640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 msgr2=0x7f98f003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/1539065682 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:35.333+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:09:38.268 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:36.348+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f98f4041d10 con 0x7f99041081b0 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:36.348+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 msgr2=0x7f98f003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:36.348+0000 7f98fb7fe640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.350+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f98f40371e0 con 0x7f99041081b0 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.350+0000 7f98fb7fe640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 0x7f98f003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.350+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f98f4047e50 con 0x7f98f003ca80 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.350+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 0x7f98f003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.351+0000 7f98fbfff640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 0x7f98f003ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98ec003980 tx=0x7f98ec006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.352+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f98f4047e50 con 0x7f98f003ca80 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.364+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9904108610 con 0x7f98f003ca80 2024-09-12T02:09:38.269 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.364+0000 7f98fb7fe640 1 -- 172.21.15.3:0/1806916451 <== mgr.14150 v2:172.21.15.3:6800/2546806763 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9904108610 con 0x7f98f003ca80 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 msgr2=0x7f98f003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 0x7f98f003ee70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98ec003980 tx=0x7f98ec006e10 comp rx=0 tx=0).stop 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 msgr2=0x7f990419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097f0 tx=0x7f98f40327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 shutdown_connections 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f98f003ca80 0x7f98f003ee70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:6800/1539065682,v1:172.21.15.3:6801/1539065682] conn(0x7f98f0038fc0 0x7f98f003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 --2- 172.21.15.3:0/1806916451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f99041081b0 0x7f990419b780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 >> 172.21.15.3:0/1806916451 conn(0x7f9904101e10 msgr2=0x7f9904102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 shutdown_connections 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:37.365+0000 7f9908aed640 1 -- 172.21.15.3:0/1806916451 wait complete. 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2024-09-12T02:09:38.270 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2024-09-12T02:09:38.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:38 smithi003 ceph-mon[31476]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:09:38.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:38 smithi003 ceph-mon[31476]: mgrmap e11: x(active, since 1.0062s) 2024-09-12T02:09:38.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:38 smithi003 ceph-mon[31476]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:09:38.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:39.490 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:39.185+0000 7fb5e2605640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:09:39] ENGINE Error in HTTPServer.serve 2024-09-12T02:09:39.490 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: Traceback (most recent call last): 2024-09-12T02:09:39.490 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:09:39.490 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._connections.run(self.expiration_interval) 2024-09-12T02:09:39.490 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._run(expiration_interval) 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: s = self.context.wrap_socket( 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: return self.sslsocket_class._create( 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:09:39.491 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self.do_handshake() 2024-09-12T02:09:39.492 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:09:39.492 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: self._sslobj.do_handshake() 2024-09-12T02:09:39.492 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:09:39.492 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:09:39.497 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad9dc4640 1 Processor -- start 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad9dc4640 1 -- start start 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad9dc4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad9dc4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4106fd0 con 0x7f8ad4106600 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48598/0 (socket says 172.21.15.3:48598) 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.718+0000 7f8ad37fe640 1 -- 172.21.15.3:0/477836151 learned_addr learned my addr 172.21.15.3:0/477836151 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.719+0000 7f8ad37fe640 1 -- 172.21.15.3:0/477836151 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad4107800 con 0x7f8ad4106600 2024-09-12T02:09:39.498 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.719+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/477836151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009820 tx=0x7f8ac402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfed492b870bced8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad27fc640 1 -- 172.21.15.3:0/477836151 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac4037070 con 0x7f8ad4106600 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad27fc640 1 -- 172.21.15.3:0/477836151 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4032920 con 0x7f8ad4106600 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad27fc640 1 -- 172.21.15.3:0/477836151 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac4030810 con 0x7f8ad4106600 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/477836151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 msgr2=0x7f8ad4106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/477836151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009820 tx=0x7f8ac402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/477836151 shutdown_connections 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/477836151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad4106a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/477836151 >> 172.21.15.3:0/477836151 conn(0x7f8ad4101e10 msgr2=0x7f8ad4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/477836151 shutdown_connections 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.720+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/477836151 wait complete. 2024-09-12T02:09:39.499 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.721+0000 7f8ad9dc4640 1 Processor -- start 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.721+0000 7f8ad9dc4640 1 -- start start 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.721+0000 7f8ad9dc4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad9dc4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410abb0 con 0x7f8ad4106600 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48604/0 (socket says 172.21.15.3:48604) 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad37fe640 1 -- 172.21.15.3:0/2265086163 learned_addr learned my addr 172.21.15.3:0/2265086163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad37fe640 1 -- 172.21.15.3:0/2265086163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad419ac20 con 0x7f8ad4106600 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.722+0000 7f8ad37fe640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40097f0 tx=0x7f8ac40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.724+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac4037040 con 0x7f8ad4106600 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.724+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4030c40 con 0x7f8ad4106600 2024-09-12T02:09:39.500 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.724+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad419aeb0 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.724+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ac403a5b0 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.724+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad419dba0 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.725+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f8ac403a7f0 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.725+0000 7f8ad0ff9640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 0x7f8abc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.725+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ac406c640 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.725+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad410ad30 con 0x7f8ad4106600 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.725+0000 7f8ad2ffd640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 0x7f8abc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.726+0000 7f8ad2ffd640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 0x7f8abc03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80098c0 tx=0x7f8ac8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:39.501 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.730+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ac40371e0 con 0x7f8ad4106600 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.932+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8ad41031c0 con 0x7f8abc038fc0 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.980+0000 7f8ad0ff9640 1 -- 172.21.15.3:0/2265086163 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8ad41031c0 con 0x7f8abc038fc0 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 msgr2=0x7f8abc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 0x7f8abc03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8ac80098c0 tx=0x7f8ac8006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 msgr2=0x7f8ad419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40097f0 tx=0x7f8ac40327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 shutdown_connections 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8abc038fc0 0x7f8abc03b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 --2- 172.21.15.3:0/2265086163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad4106600 0x7f8ad419a6e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 >> 172.21.15.3:0/2265086163 conn(0x7f8ad4101e10 msgr2=0x7f8ad4102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.982+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 shutdown_connections 2024-09-12T02:09:39.502 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:38.983+0000 7f8ad9dc4640 1 -- 172.21.15.3:0/2265086163 wait complete. 2024-09-12T02:09:39.503 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: mgrmap e12: x(active, since 2s) 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:38] ENGINE Bus STARTING 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:39] ENGINE Serving on http://172.21.15.3:8765 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:39] ENGINE Serving on https://172.21.15.3:7150 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:39] ENGINE Bus STARTED 2024-09-12T02:09:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:39 smithi003 ceph-mon[31476]: [12/Sep/2024:02:09:39] ENGINE Error in HTTPServer.serve 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: Traceback (most recent call last): 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._run(expiration_interval) 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:09:39.779 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: s = self.context.wrap_socket( 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: return self.sslsocket_class._create( 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: self.do_handshake() 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: self._sslobj.do_handshake() 2024-09-12T02:09:39.780 INFO:journalctl@ceph.mon.a.smithi003.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:09:40.983 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:40 smithi003 ceph-mon[31476]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:40.984 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$CdgyZYkLvHYlbLMEatF.R.cgIfYEOSLIJUFjxjDvx3lY9LoBm0Bfu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726106980, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T02:09:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88ca106640 1 Processor -- start 2024-09-12T02:09:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88ca106640 1 -- start start 2024-09-12T02:09:41.327 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88ca106640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88ca106640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4108100 con 0x7f88c4107730 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48612/0 (socket says 172.21.15.3:48612) 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88c37fe640 1 -- 172.21.15.3:0/2270791863 learned_addr learned my addr 172.21.15.3:0/2270791863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.896+0000 7f88c37fe640 1 -- 172.21.15.3:0/2270791863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c4108930 con 0x7f88c4107730 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.897+0000 7f88c37fe640 1 --2- 172.21.15.3:0/2270791863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009820 tx=0x7f88b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7394f9f2a55f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.897+0000 7f88c27fc640 1 -- 172.21.15.3:0/2270791863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b4037070 con 0x7f88c4107730 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.897+0000 7f88c27fc640 1 -- 172.21.15.3:0/2270791863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4032920 con 0x7f88c4107730 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.897+0000 7f88c27fc640 1 -- 172.21.15.3:0/2270791863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b4030810 con 0x7f88c4107730 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 -- 172.21.15.3:0/2270791863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 msgr2=0x7f88c4107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:41.328 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 --2- 172.21.15.3:0/2270791863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009820 tx=0x7f88b402a270 comp rx=0 tx=0).stop 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 -- 172.21.15.3:0/2270791863 shutdown_connections 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 --2- 172.21.15.3:0/2270791863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c4107b30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 -- 172.21.15.3:0/2270791863 >> 172.21.15.3:0/2270791863 conn(0x7f88c4102fa0 msgr2=0x7f88c41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 -- 172.21.15.3:0/2270791863 shutdown_connections 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 -- 172.21.15.3:0/2270791863 wait complete. 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.898+0000 7f88ca106640 1 Processor -- start 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88ca106640 1 -- start start 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88ca106640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88ca106640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88c410bce0 con 0x7f88c4107730 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48616/0 (socket says 172.21.15.3:48616) 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c37fe640 1 -- 172.21.15.3:0/4178777523 learned_addr learned my addr 172.21.15.3:0/4178777523 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:41.329 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c37fe640 1 -- 172.21.15.3:0/4178777523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c419bca0 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c37fe640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097f0 tx=0x7f88b40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b4037040 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b4030cc0 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.899+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c419bf30 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.900+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88b403a640 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.900+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c419ec20 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.900+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f88b403a8f0 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.900+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c410be60 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.901+0000 7f88c0ff9640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 0x7f88ac03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.901+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f88b406c830 con 0x7f88c4107730 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.901+0000 7f88c2ffd640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 0x7f88ac03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:41.330 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.901+0000 7f88c2ffd640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 0x7f88ac03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098c0 tx=0x7f88b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:39.905+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88b40328c0 con 0x7f88c4107730 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.099+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f88c4107bb0 con 0x7f88ac039020 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.348+0000 7f88c0ff9640 1 -- 172.21.15.3:0/4178777523 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f88c4107bb0 con 0x7f88ac039020 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 msgr2=0x7f88ac03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 0x7f88ac03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f88b80098c0 tx=0x7f88b8006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 msgr2=0x7f88c419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88b40097f0 tx=0x7f88b40327b0 comp rx=0 tx=0).stop 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 shutdown_connections 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88ac039020 0x7f88ac03b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 --2- 172.21.15.3:0/4178777523 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c4107730 0x7f88c419b760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 >> 172.21.15.3:0/4178777523 conn(0x7f88c4102fa0 msgr2=0x7f88c4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 shutdown_connections 2024-09-12T02:09:41.331 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:40.350+0000 7f88ca106640 1 -- 172.21.15.3:0/4178777523 wait complete. 2024-09-12T02:09:41.332 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-09-12T02:09:42.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:42 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3469682955' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18b5ca9640 1 Processor -- start 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18b5ca9640 1 -- start start 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18b5ca9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18b5ca9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0107050 con 0x7f18b0106680 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48624/0 (socket says 172.21.15.3:48624) 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.762+0000 7f18af7fe640 1 -- 172.21.15.3:0/2511134983 learned_addr learned my addr 172.21.15.3:0/2511134983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.763+0000 7f18af7fe640 1 -- 172.21.15.3:0/2511134983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b0107880 con 0x7f18b0106680 2024-09-12T02:09:42.450 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.763+0000 7f18af7fe640 1 --2- 172.21.15.3:0/2511134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009820 tx=0x7f18a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdea17604fde3a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18aeffd640 1 -- 172.21.15.3:0/2511134983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0037070 con 0x7f18b0106680 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18aeffd640 1 -- 172.21.15.3:0/2511134983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a0032920 con 0x7f18b0106680 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18aeffd640 1 -- 172.21.15.3:0/2511134983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0030810 con 0x7f18b0106680 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 -- 172.21.15.3:0/2511134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 msgr2=0x7f18b0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/2511134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009820 tx=0x7f18a002a270 comp rx=0 tx=0).stop 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 -- 172.21.15.3:0/2511134983 shutdown_connections 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/2511134983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b0106a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 -- 172.21.15.3:0/2511134983 >> 172.21.15.3:0/2511134983 conn(0x7f18b0101e50 msgr2=0x7f18b0104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 -- 172.21.15.3:0/2511134983 shutdown_connections 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.764+0000 7f18b5ca9640 1 -- 172.21.15.3:0/2511134983 wait complete. 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18b5ca9640 1 Processor -- start 2024-09-12T02:09:42.451 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18b5ca9640 1 -- start start 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18b5ca9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18b5ca9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18b007b0f0 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18af7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48632/0 (socket says 172.21.15.3:48632) 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.765+0000 7f18af7fe640 1 -- 172.21.15.3:0/3469682955 learned_addr learned my addr 172.21.15.3:0/3469682955 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18af7fe640 1 -- 172.21.15.3:0/3469682955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019bce0 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18af7fe640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18a0030560 tx=0x7f18a0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0037040 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019bf70 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a0040070 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a003a740 con 0x7f18b0106680 2024-09-12T02:09:42.452 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.766+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b019ec60 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.767+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f18a003a9b0 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.767+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b0106a80 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.767+0000 7f18ad7fa640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 0x7f189c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.767+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18a006c830 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.768+0000 7f18a7fff640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 0x7f189c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.768+0000 7f18a7fff640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 0x7f189c03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18980098c0 tx=0x7f1898006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.772+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18a003adf0 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.949+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f18b002d9a0 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.950+0000 7f18ad7fa640 1 -- 172.21.15.3:0/3469682955 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f18a0032920 con 0x7f18b0106680 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 msgr2=0x7f189c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 0x7f189c03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18980098c0 tx=0x7f1898006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 msgr2=0x7f18b019b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:42.453 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18a0030560 tx=0x7f18a0032670 comp rx=0 tx=0).stop 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 shutdown_connections 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f189c038fc0 0x7f189c03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 --2- 172.21.15.3:0/3469682955 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18b0106680 0x7f18b019b7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 >> 172.21.15.3:0/3469682955 conn(0x7f18b0101e50 msgr2=0x7f18b0102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 shutdown_connections 2024-09-12T02:09:42.454 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:41.953+0000 7f18b5ca9640 1 -- 172.21.15.3:0/3469682955 wait complete. 2024-09-12T02:09:42.458 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:09:42.458 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-09-12T02:09:42.465 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:09:42.465 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-09-12T02:09:42.465 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-09-12T02:09:42.466 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout: Password: ugzu0c05l1 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:42.468 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config directory 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.863+0000 7f9a756a1640 1 Processor -- start 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.863+0000 7f9a756a1640 1 -- start start 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.863+0000 7f9a756a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.863+0000 7f9a756a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7007bdd0 con 0x7f9a701081f0 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.863+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:43.716 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.864+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48638/0 (socket says 172.21.15.3:48638) 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.864+0000 7f9a6effd640 1 -- 172.21.15.3:0/1887322991 learned_addr learned my addr 172.21.15.3:0/1887322991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.864+0000 7f9a6effd640 1 -- 172.21.15.3:0/1887322991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7010ab20 con 0x7f9a701081f0 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a6effd640 1 --2- 172.21.15.3:0/1887322991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009820 tx=0x7f9a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473701e83be194b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a6dffb640 1 -- 172.21.15.3:0/1887322991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a68037070 con 0x7f9a701081f0 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a6dffb640 1 -- 172.21.15.3:0/1887322991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68032920 con 0x7f9a701081f0 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a6dffb640 1 -- 172.21.15.3:0/1887322991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a68030810 con 0x7f9a701081f0 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 -- 172.21.15.3:0/1887322991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 msgr2=0x7f9a7010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 --2- 172.21.15.3:0/1887322991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009820 tx=0x7f9a6802a270 comp rx=0 tx=0).stop 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 -- 172.21.15.3:0/1887322991 shutdown_connections 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 --2- 172.21.15.3:0/1887322991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a7010a5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 -- 172.21.15.3:0/1887322991 >> 172.21.15.3:0/1887322991 conn(0x7f9a70101e10 msgr2=0x7f9a70104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:43.717 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 -- 172.21.15.3:0/1887322991 shutdown_connections 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.865+0000 7f9a756a1640 1 -- 172.21.15.3:0/1887322991 wait complete. 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.866+0000 7f9a756a1640 1 Processor -- start 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.866+0000 7f9a756a1640 1 -- start start 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.866+0000 7f9a756a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.866+0000 7f9a756a1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010bb40 con 0x7f9a701081f0 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48648/0 (socket says 172.21.15.3:48648) 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a6effd640 1 -- 172.21.15.3:0/2982833859 learned_addr learned my addr 172.21.15.3:0/2982833859 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a6effd640 1 -- 172.21.15.3:0/2982833859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70107fb0 con 0x7f9a701081f0 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a6effd640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a68030420 tx=0x7f9a68032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a68037040 con 0x7f9a701081f0 2024-09-12T02:09:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701061a0 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a68040070 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a6803a6e0 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.867+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701066e0 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.868+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f9a6803a9e0 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.868+0000 7f9a67fff640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 0x7f9a5c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.868+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a6806c760 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.868+0000 7f9a6e7fc640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 0x7f9a5c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.868+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70108650 con 0x7f9a701081f0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.869+0000 7f9a6e7fc640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 0x7f9a5c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a580098c0 tx=0x7f9a58006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:43.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:42.873+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a68042e10 con 0x7f9a701081f0 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.003+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9a6803ad70 con 0x7f9a701081f0 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.143+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9a70106d60 con 0x7f9a701081f0 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.146+0000 7f9a67fff640 1 -- 172.21.15.3:0/2982833859 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f9a68043b80 con 0x7f9a701081f0 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 msgr2=0x7f9a5c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 0x7f9a5c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9a580098c0 tx=0x7f9a58006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 msgr2=0x7f9a70107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a68030420 tx=0x7f9a68032670 comp rx=0 tx=0).stop 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 shutdown_connections 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a5c038fc0 0x7f9a5c03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 --2- 172.21.15.3:0/2982833859 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a701081f0 0x7f9a70107a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 >> 172.21.15.3:0/2982833859 conn(0x7f9a70101e10 msgr2=0x7f9a70102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:43.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 shutdown_connections 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-09-12T02:09:43.148+0000 7f9a756a1640 1 -- 172.21.15.3:0/2982833859 wait complete. 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T02:09:43.721 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:43.722 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-09-12T02:09:43.761 INFO:tasks.cephadm:Fetching config... 2024-09-12T02:09:43.762 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:09:43.762 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T02:09:43.780 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T02:09:43.780 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:09:43.781 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T02:09:43.838 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T02:09:43.838 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:09:43.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T02:09:43.906 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T02:09:43.906 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:09:43.906 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T02:09:43.963 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T02:09:43.963 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:09:44.041 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:44.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:09:44.096 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:44.109 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:09:44.149 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCecC5fQNLpcE1bk4NvDEuNBBAyMt0XeOrIULwezP/hWf7zbDA6RiXSPuwHkck9194i6YtVFRk/YgwOVFdd4AneVJ1mhTmApzhaF9P2ANIt2CzaIMYCKEnizP4ZoCNKm0k0NSZshV/I+UfnWEk9EwwvB/8xKNBppbKvhG/ZxdbqWCwYABoL7IibAnSLOzltr67BABlrVPzdx6dxoNHsCnqmeKBjlpBlKgdLzpq9Sd2D3pWUOnb+HYlWp8nQaFqqcj+wgxRYB7jK7QqgmS9htY0tNnpmrio/nVSkklLrRRkb5CJwF8JMp4WID2ITBkD3qn7vCiCjGxfBpM3SEhVblAlC4kjHSiUNv3GSjsDeqrULeVvmmkOgMSRe6J/UCP8UzKDn/baLjAxgfvE0ptxHH/dTuQTOtziQkqg26l54QQM9dRAixiznrgiqPVLEBCrb+4Bxjo/MB95nXBBeysj/6cYjxNSICsawD4Fh2EqsTBtT+VOo4bvGy9OOgn2OTmMDgjU= ceph-c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:09:44.162 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T02:09:44.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:44 smithi003 ceph-mon[31476]: mgrmap e13: x(active, since 6s) 2024-09-12T02:09:44.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:44 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2982833859' entity='client.admin' 2024-09-12T02:09:44.417 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.837+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/3468480736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 msgr2=0x7ff8ec101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.837+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/3468480736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009880 tx=0x7ff8e802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.837+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/3468480736 shutdown_connections 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.837+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/3468480736 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.837+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/3468480736 >> 172.21.15.3:0/3468480736 conn(0x7ff8ec0fcdb0 msgr2=0x7ff8ec0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/3468480736 shutdown_connections 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/3468480736 wait complete. 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 Processor -- start 2024-09-12T02:09:44.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 -- start start 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f3ee5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec107710 con 0x7ff8ec101600 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.838+0000 7ff8f1c81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f1c81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48664/0 (socket says 172.21.15.3:48664) 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f1c81640 1 -- 172.21.15.3:0/16542952 learned_addr learned my addr 172.21.15.3:0/16542952 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f1c81640 1 -- 172.21.15.3:0/16542952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec1972f0 con 0x7ff8ec101600 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f1c81640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004f60 tx=0x7ff8e80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e803f070 con 0x7ff8ec101600 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8004d40 con 0x7ff8ec101600 2024-09-12T02:09:44.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec197580 con 0x7ff8ec101600 2024-09-12T02:09:44.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e8035030 con 0x7ff8ec101600 2024-09-12T02:09:44.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.839+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec19a230 con 0x7ff8ec101600 2024-09-12T02:09:44.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.840+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff8e8042070 con 0x7ff8ec101600 2024-09-12T02:09:44.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.840+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec107e90 con 0x7ff8ec101600 2024-09-12T02:09:44.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.840+0000 7ff8e2ffd640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 0x7ff8d803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:44.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.841+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8e806cdc0 con 0x7ff8ec101600 2024-09-12T02:09:44.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.841+0000 7ff8f1480640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 0x7ff8d803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:44.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.841+0000 7ff8f1480640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 0x7ff8d803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098c0 tx=0x7ff8dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:44.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:44.845+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8e803a3c0 con 0x7ff8ec101600 2024-09-12T02:09:45.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.022+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff8ec02d940 con 0x7ff8ec101600 2024-09-12T02:09:45.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.025+0000 7ff8e2ffd640 1 -- 172.21.15.3:0/16542952 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff8e8003dc0 con 0x7ff8ec101600 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 msgr2=0x7ff8d803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 0x7ff8d803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098c0 tx=0x7ff8dc006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 msgr2=0x7ff8ec196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004f60 tx=0x7ff8e80048f0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 shutdown_connections 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff8d8039060 0x7ff8d803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 --2- 172.21.15.3:0/16542952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8ec101600 0x7ff8ec196db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:45.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 >> 172.21.15.3:0/16542952 conn(0x7ff8ec0fcdb0 msgr2=0x7ff8ec0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:45.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 shutdown_connections 2024-09-12T02:09:45.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:45.027+0000 7ff8f3ee5640 1 -- 172.21.15.3:0/16542952 wait complete. 2024-09-12T02:09:46.168 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T02:09:46.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T02:09:46.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:46 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/16542952' entity='client.admin' 2024-09-12T02:09:46.607 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 -- 172.21.15.3:0/13179748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 msgr2=0x7f01881019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 --2- 172.21.15.3:0/13179748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f01881019e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0178009c20 tx=0x7f017802aa90 comp rx=0 tx=0).stop 2024-09-12T02:09:47.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 -- 172.21.15.3:0/13179748 shutdown_connections 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 --2- 172.21.15.3:0/13179748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f01881019e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 -- 172.21.15.3:0/13179748 >> 172.21.15.3:0/13179748 conn(0x7f01880fcd70 msgr2=0x7f01880ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 -- 172.21.15.3:0/13179748 shutdown_connections 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.051+0000 7f018d35c640 1 -- 172.21.15.3:0/13179748 wait complete. 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f018d35c640 1 Processor -- start 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f018d35c640 1 -- start start 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f018d35c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f018d35c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01881076f0 con 0x7f01881015e0 2024-09-12T02:09:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f0186ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f0186ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48692/0 (socket says 172.21.15.3:48692) 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f0186ffd640 1 -- 172.21.15.3:0/2052914348 learned_addr learned my addr 172.21.15.3:0/2052914348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.052+0000 7f0186ffd640 1 -- 172.21.15.3:0/2052914348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01881994f0 con 0x7f01881015e0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f0186ffd640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0178005cc0 tx=0x7f017802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0178047070 con 0x7f01881015e0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0178003e50 con 0x7f01881015e0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0178033050 con 0x7f01881015e0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0188199780 con 0x7f01881015e0 2024-09-12T02:09:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.053+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018819c470 con 0x7f01881015e0 2024-09-12T02:09:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.054+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0188107e70 con 0x7f01881015e0 2024-09-12T02:09:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.054+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0178036070 con 0x7f01881015e0 2024-09-12T02:09:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.054+0000 7f0184ff9640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 0x7f017403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:47.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.055+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f017803e030 con 0x7f01881015e0 2024-09-12T02:09:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.055+0000 7f017ffff640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 0x7f017403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:47.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.055+0000 7f017ffff640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 0x7f017403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01700098c0 tx=0x7f0170006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:47.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.059+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0178041cd0 con 0x7f01881015e0 2024-09-12T02:09:47.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.242+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0188062630 con 0x7f01740390c0 2024-09-12T02:09:47.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.246+0000 7f0184ff9640 1 -- 172.21.15.3:0/2052914348 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0188062630 con 0x7f01740390c0 2024-09-12T02:09:47.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 msgr2=0x7f017403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 0x7f017403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01700098c0 tx=0x7f0170006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 msgr2=0x7f0188198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0178005cc0 tx=0x7f017802fc10 comp rx=0 tx=0).stop 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 shutdown_connections 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01740390c0 0x7f017403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 --2- 172.21.15.3:0/2052914348 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01881015e0 0x7f0188198fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.247+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 >> 172.21.15.3:0/2052914348 conn(0x7f01880fcd70 msgr2=0x7f01880fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.248+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 shutdown_connections 2024-09-12T02:09:47.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:47.248+0000 7f018d35c640 1 -- 172.21.15.3:0/2052914348 wait complete. 2024-09-12T02:09:47.931 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi026 2024-09-12T02:09:47.932 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:09:47.932 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:09:47.950 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:09:47.951 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:09:48.009 INFO:tasks.cephadm:Adding host smithi026 to orchestrator... 2024-09-12T02:09:48.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch host add smithi026 2024-09-12T02:09:48.345 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:48.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:48 smithi003 ceph-mon[31476]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:48.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 -- 172.21.15.3:0/818970175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 msgr2=0x7fd688104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 --2- 172.21.15.3:0/818970175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688104520 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd684009880 tx=0x7fd68402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 -- 172.21.15.3:0/818970175 shutdown_connections 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 --2- 172.21.15.3:0/818970175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688104520 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 -- 172.21.15.3:0/818970175 >> 172.21.15.3:0/818970175 conn(0x7fd6880fbcd0 msgr2=0x7fd6880fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 -- 172.21.15.3:0/818970175 shutdown_connections 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.835+0000 7fd68faa5640 1 -- 172.21.15.3:0/818970175 wait complete. 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68faa5640 1 Processor -- start 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68faa5640 1 -- start start 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68faa5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68faa5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881076e0 con 0x7fd688102130 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68d841640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68d841640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48448/0 (socket says 172.21.15.3:48448) 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68d841640 1 -- 172.21.15.3:0/708752075 learned_addr learned my addr 172.21.15.3:0/708752075 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68d841640 1 -- 172.21.15.3:0/708752075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6881972e0 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.836+0000 7fd68d841640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd684004f60 tx=0x7fd6840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.837+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd68403f070 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.837+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684003ad0 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.837+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd684035030 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.837+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd688197570 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.837+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68819a220 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.838+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd684042070 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.838+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688107e60 con 0x7fd688102130 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.838+0000 7fd67effd640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.838+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd68406ce60 con 0x7fd688102130 2024-09-12T02:09:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.838+0000 7fd68d040640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.839+0000 7fd68d040640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 0x7fd67403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6780098c0 tx=0x7fd678006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:48.843+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd68404c350 con 0x7fd688102130 2024-09-12T02:09:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:49.030+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}) -- 0x7fd68802d8a0 con 0x7fd6740390c0 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:50.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:50 smithi003 ceph-mon[31476]: Updating smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:09:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:50 smithi003 ceph-mon[31476]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:09:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:50 smithi003 ceph-mon[31476]: Updating smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:09:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:50 smithi003 ceph-mon[31476]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:51.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:51 smithi003 ceph-mon[31476]: Deploying cephadm binary to smithi026 2024-09-12T02:09:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.937+0000 7fd67effd640 1 -- 172.21.15.3:0/708752075 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd68802d8a0 con 0x7fd6740390c0 2024-09-12T02:09:52.937 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi026' with addr '172.21.15.26' 2024-09-12T02:09:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 msgr2=0x7fd67403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 0x7fd67403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6780098c0 tx=0x7fd678006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:52.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 msgr2=0x7fd688196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd684004f60 tx=0x7fd6840048f0 comp rx=0 tx=0).stop 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 shutdown_connections 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6740390c0 0x7fd67403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 --2- 172.21.15.3:0/708752075 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd688102130 0x7fd688196da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 >> 172.21.15.3:0/708752075 conn(0x7fd6880fbcd0 msgr2=0x7fd6880fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 shutdown_connections 2024-09-12T02:09:52.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:52.939+0000 7fd68faa5640 1 -- 172.21.15.3:0/708752075 wait complete. 2024-09-12T02:09:53.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:09:53.751 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:54.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:54.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:53 smithi003 ceph-mon[31476]: Added host smithi026 2024-09-12T02:09:54.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:09:54.181 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:54.182 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 -- 172.21.15.3:0/3087818583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 msgr2=0x7f4860102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 --2- 172.21.15.3:0/3087818583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f4860102b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4850009880 tx=0x7f485002a670 comp rx=0 tx=0).stop 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 -- 172.21.15.3:0/3087818583 shutdown_connections 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 --2- 172.21.15.3:0/3087818583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f4860102b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 -- 172.21.15.3:0/3087818583 >> 172.21.15.3:0/3087818583 conn(0x7f48600fc1b0 msgr2=0x7f48600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 -- 172.21.15.3:0/3087818583 shutdown_connections 2024-09-12T02:09:54.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.189+0000 7f48650ef640 1 -- 172.21.15.3:0/3087818583 wait complete. 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f48650ef640 1 Processor -- start 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f48650ef640 1 -- start start 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f48650ef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f48650ef640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f486010bcc0 con 0x7f4860102750 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f485ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f485ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48454/0 (socket says 172.21.15.3:48454) 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.190+0000 7f485ed9d640 1 -- 172.21.15.3:0/2572343520 learned_addr learned my addr 172.21.15.3:0/2572343520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f485ed9d640 1 -- 172.21.15.3:0/2572343520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4860197710 con 0x7f4860102750 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f485ed9d640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4850009230 tx=0x7f4850003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:54.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f485003f070 con 0x7f4860102750 2024-09-12T02:09:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f485003bd40 con 0x7f4860102750 2024-09-12T02:09:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48601979a0 con 0x7f4860102750 2024-09-12T02:09:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4850035030 con 0x7f4860102750 2024-09-12T02:09:54.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.191+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4860197e00 con 0x7f4860102750 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.192+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4850042070 con 0x7f4860102750 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.192+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486006a7b0 con 0x7f4860102750 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.192+0000 7f4847fff640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 0x7f484803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.192+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f485006cc40 con 0x7f4860102750 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.192+0000 7f485e59c640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 0x7f484803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:54.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.193+0000 7f485e59c640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 0x7f484803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48540098c0 tx=0x7f4854006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:54.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.197+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f485003a890 con 0x7f4860102750 2024-09-12T02:09:54.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.379+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4860198530 con 0x7f4848039060 2024-09-12T02:09:54.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.380+0000 7f4847fff640 1 -- 172.21.15.3:0/2572343520 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f4860198530 con 0x7f4848039060 2024-09-12T02:09:54.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:09:54.380 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}] 2024-09-12T02:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 msgr2=0x7f484803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 0x7f484803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f48540098c0 tx=0x7f4854006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 msgr2=0x7f48601971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4850009230 tx=0x7f4850003d80 comp rx=0 tx=0).stop 2024-09-12T02:09:54.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 shutdown_connections 2024-09-12T02:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4848039060 0x7f484803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 --2- 172.21.15.3:0/2572343520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4860102750 0x7f48601971d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 >> 172.21.15.3:0/2572343520 conn(0x7f48600fc1b0 msgr2=0x7f4860105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 shutdown_connections 2024-09-12T02:09:54.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:54.382+0000 7f48650ef640 1 -- 172.21.15.3:0/2572343520 wait complete. 2024-09-12T02:09:54.947 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-09-12T02:09:54.947 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:09:54.947 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:09:54.967 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:09:54.967 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:09:55.025 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-09-12T02:09:55.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch host add smithi092 2024-09-12T02:09:55.275 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:09:55.548 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 -- 172.21.15.3:0/3518888783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 msgr2=0x7f351c102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 --2- 172.21.15.3:0/3518888783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c102b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3518009880 tx=0x7f351802a670 comp rx=0 tx=0).stop 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 -- 172.21.15.3:0/3518888783 shutdown_connections 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 --2- 172.21.15.3:0/3518888783 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c102b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 -- 172.21.15.3:0/3518888783 >> 172.21.15.3:0/3518888783 conn(0x7f351c0fc1b0 msgr2=0x7f351c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:55.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 -- 172.21.15.3:0/3518888783 shutdown_connections 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.746+0000 7f352367b640 1 -- 172.21.15.3:0/3518888783 wait complete. 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f352367b640 1 Processor -- start 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f352367b640 1 -- start start 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f352367b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f352367b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10bcc0 con 0x7f351c102750 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f3521417640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f3521417640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48478/0 (socket says 172.21.15.3:48478) 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f3521417640 1 -- 172.21.15.3:0/174544263 learned_addr learned my addr 172.21.15.3:0/174544263 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:09:55.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.747+0000 7f3521417640 1 -- 172.21.15.3:0/174544263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c197710 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f3521417640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3518009230 tx=0x7f3518003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f351803f070 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351803bc80 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1979a0 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3518035030 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.748+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c197e00 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3518042070 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351c06a7b0 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f35127fc640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 0x7f350803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f351806c960 con 0x7f351c102750 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f3520c16640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 0x7f350803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:09:55.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.749+0000 7f3520c16640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 0x7f350803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f350c0098c0 tx=0x7f350c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:09:55.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.754+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35180314c0 con 0x7f351c102750 2024-09-12T02:09:55.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:55.937+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) -- 0x7f351c198530 con 0x7f35080390c0 2024-09-12T02:09:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:56 smithi003 ceph-mon[31476]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:09:57.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:09:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:09:58.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: Deploying cephadm binary to smithi092 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: Updating smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:09:59.679 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:09:59 smithi003 ceph-mon[31476]: Updating smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:09:59.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.755+0000 7f35127fc640 1 -- 172.21.15.3:0/174544263 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f351c198530 con 0x7f35080390c0 2024-09-12T02:09:59.756 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-09-12T02:09:59.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 msgr2=0x7f350803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 0x7f350803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f350c0098c0 tx=0x7f350c006db0 comp rx=0 tx=0).stop 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 msgr2=0x7f351c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3518009230 tx=0x7f3518003d40 comp rx=0 tx=0).stop 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 shutdown_connections 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f35080390c0 0x7f350803b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 --2- 172.21.15.3:0/174544263 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f351c102750 0x7f351c1971d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 >> 172.21.15.3:0/174544263 conn(0x7f351c0fc1b0 msgr2=0x7f351c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 shutdown_connections 2024-09-12T02:09:59.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:09:59.757+0000 7f352367b640 1 -- 172.21.15.3:0/174544263 wait complete. 2024-09-12T02:10:00.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:00 smithi003 ceph-mon[31476]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:00.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:00.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:00.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:00.539 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:10:00.791 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:01.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 -- 172.21.15.3:0/1338738621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 msgr2=0x7fb1ec106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 --2- 172.21.15.3:0/1338738621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009880 tx=0x7fb1e802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 -- 172.21.15.3:0/1338738621 shutdown_connections 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 --2- 172.21.15.3:0/1338738621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 -- 172.21.15.3:0/1338738621 >> 172.21.15.3:0/1338738621 conn(0x7fb1ec0fc1b0 msgr2=0x7fb1ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.229+0000 7fb1f370d640 1 -- 172.21.15.3:0/1338738621 shutdown_connections 2024-09-12T02:10:01.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.230+0000 7fb1f370d640 1 -- 172.21.15.3:0/1338738621 wait complete. 2024-09-12T02:10:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.230+0000 7fb1f370d640 1 Processor -- start 2024-09-12T02:10:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.230+0000 7fb1f370d640 1 -- start start 2024-09-12T02:10:01.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.230+0000 7fb1f370d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.230+0000 7fb1f370d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10ab90 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f14a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f14a9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45998/0 (socket says 172.21.15.3:45998) 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f14a9640 1 -- 172.21.15.3:0/3727255239 learned_addr learned my addr 172.21.15.3:0/3727255239 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f14a9640 1 -- 172.21.15.3:0/3727255239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec1976a0 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f14a9640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009230 tx=0x7fb1e8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1e803f070 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e803bc80 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec197930 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1e8035030 con 0x7fb1ec106560 2024-09-12T02:10:01.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.231+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec19ba30 con 0x7fb1ec106560 2024-09-12T02:10:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.232+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb1e8042070 con 0x7fb1ec106560 2024-09-12T02:10:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.232+0000 7fb1e27fc640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 0x7fb1d803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.232+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec101bd0 con 0x7fb1ec106560 2024-09-12T02:10:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.232+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1e806cdb0 con 0x7fb1ec106560 2024-09-12T02:10:01.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.232+0000 7fb1f0ca8640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 0x7fb1d803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:01.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.233+0000 7fb1f0ca8640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 0x7fb1d803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0098c0 tx=0x7fb1dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:01.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.237+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1e803ac60 con 0x7fb1ec106560 2024-09-12T02:10:01.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.420+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1ec1984c0 con 0x7fb1d8039060 2024-09-12T02:10:01.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.421+0000 7fb1e27fc640 1 -- 172.21.15.3:0/3727255239 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+237 (secure 0 0 0) 0x7fb1ec1984c0 con 0x7fb1d8039060 2024-09-12T02:10:01.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:10:01.422 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-09-12T02:10:01.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 msgr2=0x7fb1d803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:01.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 0x7fb1d803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0098c0 tx=0x7fb1dc006db0 comp rx=0 tx=0).stop 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 msgr2=0x7fb1ec197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009230 tx=0x7fb1e8003d40 comp rx=0 tx=0).stop 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 shutdown_connections 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb1d8039060 0x7fb1d803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 --2- 172.21.15.3:0/3727255239 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1ec106560 0x7fb1ec197160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 >> 172.21.15.3:0/3727255239 conn(0x7fb1ec0fc1b0 msgr2=0x7fb1ec069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 shutdown_connections 2024-09-12T02:10:01.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:01.423+0000 7fb1f370d640 1 -- 172.21.15.3:0/3727255239 wait complete. 2024-09-12T02:10:01.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:01 smithi003 ceph-mon[31476]: Added host smithi092 2024-09-12T02:10:01.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:01.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:02.003 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T02:10:02.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T02:10:02.461 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:02.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:02 smithi003 ceph-mon[31476]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1233975164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 msgr2=0x7ff4cc107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1233975164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc107a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009880 tx=0x7ff4c002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1233975164 shutdown_connections 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1233975164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc107a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1233975164 >> 172.21.15.3:0/1233975164 conn(0x7ff4cc0fd260 msgr2=0x7ff4cc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1233975164 shutdown_connections 2024-09-12T02:10:02.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1233975164 wait complete. 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 Processor -- start 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.911+0000 7ff4d2d6a640 1 -- start start 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d2d6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d2d6a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc10bc80 con 0x7ff4cc107650 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d0b06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d0b06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46014/0 (socket says 172.21.15.3:46014) 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d0b06640 1 -- 172.21.15.3:0/1504885953 learned_addr learned my addr 172.21.15.3:0/1504885953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d0b06640 1 -- 172.21.15.3:0/1504885953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc197660 con 0x7ff4cc107650 2024-09-12T02:10:02.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d0b06640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009230 tx=0x7ff4c0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c003f070 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc1978f0 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c003bc80 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.912+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4c0035030 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.913+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc19b9f0 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ff4c0042070 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc102cc0 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4c9ffb640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 0x7ff4bc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff4c006cd40 con 0x7ff4cc107650 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4cbfff640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 0x7ff4bc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:02.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.914+0000 7ff4cbfff640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 0x7ff4bc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80098c0 tx=0x7ff4b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:02.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:02.919+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4c00372d0 con 0x7ff4cc107650 2024-09-12T02:10:03.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.101+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff4cc107ab0 con 0x7ff4cc107650 2024-09-12T02:10:03.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.294+0000 7ff4c9ffb640 1 -- 172.21.15.3:0/1504885953 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff4c0031570 con 0x7ff4cc107650 2024-09-12T02:10:03.296 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 msgr2=0x7ff4bc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 0x7ff4bc03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80098c0 tx=0x7ff4b8006db0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 msgr2=0x7ff4cc197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009230 tx=0x7ff4c0003d40 comp rx=0 tx=0).stop 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 shutdown_connections 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff4bc039060 0x7ff4bc03b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 --2- 172.21.15.3:0/1504885953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4cc107650 0x7ff4cc197120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:03.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 >> 172.21.15.3:0/1504885953 conn(0x7ff4cc0fd260 msgr2=0x7ff4cc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:03.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 shutdown_connections 2024-09-12T02:10:03.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:03.297+0000 7ff4d2d6a640 1 -- 172.21.15.3:0/1504885953 wait complete. 2024-09-12T02:10:03.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:03 smithi003 ceph-mon[31476]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:10:03.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:03 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1504885953' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:10:03.818 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-09-12T02:10:03.818 INFO:tasks.cephadm:Adding mon.b on smithi026 2024-09-12T02:10:03.818 INFO:tasks.cephadm:Adding mon.c on smithi092 2024-09-12T02:10:03.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c' 2024-09-12T02:10:04.066 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:10:04.111 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/2429488691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 msgr2=0x7f8d580ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/2429488691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d580ffcd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009880 tx=0x7f8d4c02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/2429488691 shutdown_connections 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/2429488691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d580ffcd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/2429488691 >> 172.21.15.92:0/2429488691 conn(0x7f8d580fb400 msgr2=0x7f8d580fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/2429488691 shutdown_connections 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.585+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/2429488691 wait complete. 2024-09-12T02:10:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d5dbb4640 1 Processor -- start 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d5dbb4640 1 -- start start 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d5dbb4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d5dbb4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8d58107720 con 0x7f8d580ff8d0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.92:56342/0 (socket says 172.21.15.92:56342) 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.586+0000 7f8d577fe640 1 -- 172.21.15.92:0/4005977325 learned_addr learned my addr 172.21.15.92:0/4005977325 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d577fe640 1 -- 172.21.15.92:0/4005977325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d58197370 con 0x7f8d580ff8d0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d577fe640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004f60 tx=0x7f8d4c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d4c03f070 con 0x7f8d580ff8d0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c004d40 con 0x7f8d580ff8d0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d58197600 con 0x7f8d580ff8d0 2024-09-12T02:10:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d4c035030 con 0x7f8d580ff8d0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.587+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5819a2f0 con 0x7f8d580ff8d0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.588+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f8d4c042070 con 0x7f8d580ff8d0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.588+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d58107ea0 con 0x7f8d580ff8d0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.588+0000 7f8d54ff9640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 0x7f8d4003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.588+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8d4c06cda0 con 0x7f8d580ff8d0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.588+0000 7f8d56ffd640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 0x7f8d4003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.589+0000 7f8d56ffd640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 0x7f8d4003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098c0 tx=0x7f8d48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:04.627 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.593+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d4c040260 con 0x7f8d580ff8d0 2024-09-12T02:10:04.781 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1504885953' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:04.782 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:04 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:04.783 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.779+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c", "target": ["mon-mgr", ""]}) -- 0x7f8d580626e0 con 0x7f8d40039060 2024-09-12T02:10:04.785 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.785+0000 7f8d54ff9640 1 -- 172.21.15.92:0/4005977325 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8d580626e0 con 0x7f8d40039060 2024-09-12T02:10:04.785 INFO:teuthology.orchestra.run.smithi092.stdout:Scheduled mon update... 2024-09-12T02:10:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 msgr2=0x7f8d4003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 0x7f8d4003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098c0 tx=0x7f8d48006db0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.787 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 msgr2=0x7f8d58196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004f60 tx=0x7f8d4c0048f0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 shutdown_connections 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8d40039060 0x7f8d4003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 --2- 172.21.15.92:0/4005977325 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8d580ff8d0 0x7f8d58196e30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 >> 172.21.15.92:0/4005977325 conn(0x7f8d580fb400 msgr2=0x7f8d580fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 shutdown_connections 2024-09-12T02:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:04.787+0000 7f8d5dbb4640 1 -- 172.21.15.92:0/4005977325 wait complete. 2024-09-12T02:10:05.364 DEBUG:teuthology.orchestra.run.smithi026:mon.b> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T02:10:05.367 DEBUG:teuthology.orchestra.run.smithi092:mon.c> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T02:10:05.369 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:10:05.370 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:10:05.516 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:05.517 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:05.518 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:05.643 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 -- 172.21.15.92:0/4068857568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 msgr2=0x7f5e24107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 --2- 172.21.15.92:0/4068857568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e24107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009880 tx=0x7f5e1802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 -- 172.21.15.92:0/4068857568 shutdown_connections 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 --2- 172.21.15.92:0/4068857568 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e24107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 -- 172.21.15.92:0/4068857568 >> 172.21.15.92:0/4068857568 conn(0x7f5e240fd290 msgr2=0x7f5e240ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 -- 172.21.15.92:0/4068857568 shutdown_connections 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.113+0000 7f5e2b149640 1 -- 172.21.15.92:0/4068857568 wait complete. 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e2b149640 1 Processor -- start 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e2b149640 1 -- start start 2024-09-12T02:10:06.115 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e2b149640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e2b149640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2410bc90 con 0x7f5e24107660 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e28ee5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e28ee5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.92:56356/0 (socket says 172.21.15.92:56356) 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.114+0000 7f5e28ee5640 1 -- 172.21.15.92:0/1169635911 learned_addr learned my addr 172.21.15.92:0/1169635911 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e28ee5640 1 -- 172.21.15.92:0/1169635911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e240763e0 con 0x7f5e24107660 2024-09-12T02:10:06.116 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e28ee5640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009230 tx=0x7f5e18003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:06.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e1803f070 con 0x7f5e24107660 2024-09-12T02:10:06.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e24074a30 con 0x7f5e24107660 2024-09-12T02:10:06.117 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1803bc80 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e18035030 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.115+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e24074ef0 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.116+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5e18042070 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.116+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e24102cd0 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.116+0000 7f5e21ffb640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 0x7f5e1403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.116+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5e1806cce0 con 0x7f5e24107660 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.117+0000 7f5e23fff640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 0x7f5e1403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.117+0000 7f5e23fff640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 0x7f5e1403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098c0 tx=0x7f5e10006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:06.122 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.121+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e1804c350 con 0x7f5e24107660 2024-09-12T02:10:06.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.364+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5e24075a20 con 0x7f5e24107660 2024-09-12T02:10:06.365 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.364+0000 7f5e21ffb640 1 -- 172.21.15.92:0/1169635911 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f5e18031e20 con 0x7f5e24107660 2024-09-12T02:10:06.365 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:10:06.366 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","modified":"2024-09-12T02:08:36.454578Z","created":"2024-09-12T02:08:36.454578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:10:06.366 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 1 2024-09-12T02:10:06.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 msgr2=0x7f5e1403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 0x7f5e1403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098c0 tx=0x7f5e10006db0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 msgr2=0x7f5e240744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:06.367 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009230 tx=0x7f5e18003d40 comp rx=0 tx=0).stop 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 shutdown_connections 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e14039060 0x7f5e1403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 --2- 172.21.15.92:0/1169635911 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e24107660 0x7f5e240744f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 >> 172.21.15.92:0/1169635911 conn(0x7f5e240fd290 msgr2=0x7f5e240feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 shutdown_connections 2024-09-12T02:10:06.368 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:06.367+0000 7f5e2b149640 1 -- 172.21.15.92:0/1169635911 wait complete. 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c;count:3 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: Reconfiguring daemon mon.a on smithi003 2024-09-12T02:10:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:06.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:07.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:07 smithi003 ceph-mon[31476]: Deploying daemon mon.c on smithi092 2024-09-12T02:10:07.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:07 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/1169635911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:07.982 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:10:07.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:10:08.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:08 smithi003 ceph-mon[31476]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:10:09.678 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 -- 172.21.15.92:0/2431120839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 msgr2=0x7f88e8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 --2- 172.21.15.92:0/2431120839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e8100830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009880 tx=0x7f88d802a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 -- 172.21.15.92:0/2431120839 shutdown_connections 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 --2- 172.21.15.92:0/2431120839 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e8100830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 -- 172.21.15.92:0/2431120839 >> 172.21.15.92:0/2431120839 conn(0x7f88e80fc2c0 msgr2=0x7f88e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 -- 172.21.15.92:0/2431120839 shutdown_connections 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.677+0000 7f88ed346640 1 -- 172.21.15.92:0/2431120839 wait complete. 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88ed346640 1 Processor -- start 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88ed346640 1 -- start start 2024-09-12T02:10:09.679 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88ed346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:09.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88ed346640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810dfb0 con 0x7f88e8100450 2024-09-12T02:10:09.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88e6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:09.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88e6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.92:56368/0 (socket says 172.21.15.92:56368) 2024-09-12T02:10:09.680 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.678+0000 7f88e6ffd640 1 -- 172.21.15.92:0/1247341257 learned_addr learned my addr 172.21.15.92:0/1247341257 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:09.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88e6ffd640 1 -- 172.21.15.92:0/1247341257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8199a20 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88e6ffd640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009230 tx=0x7f88d8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d803f070 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e8199cb0 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d803bc80 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d8035030 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.679+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e819a110 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.680+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f88d8042070 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.680+0000 7f88cffff640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 0x7f88d003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.680+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e806a480 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.680+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f88d806cd80 con 0x7f88e8100450 2024-09-12T02:10:09.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.681+0000 7f88e67fc640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 0x7f88d003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:09.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.681+0000 7f88e67fc640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 0x7f88d003b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098c0 tx=0x7f88dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:09.686 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.686+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88d804c350 con 0x7f88e8100450 2024-09-12T02:10:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.952+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f88e819a840 con 0x7f88e8100450 2024-09-12T02:10:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.953+0000 7f88cffff640 1 -- 172.21.15.92:0/1247341257 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f88d804ad50 con 0x7f88e8100450 2024-09-12T02:10:09.954 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:10:09.954 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":1,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","modified":"2024-09-12T02:08:36.454578Z","created":"2024-09-12T02:08:36.454578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:10:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 1 2024-09-12T02:10:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 msgr2=0x7f88d003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 0x7f88d003b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f88dc0098c0 tx=0x7f88dc006db0 comp rx=0 tx=0).stop 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 msgr2=0x7f88e81994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009230 tx=0x7f88d8003d40 comp rx=0 tx=0).stop 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 shutdown_connections 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88d0039060 0x7f88d003b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 --2- 172.21.15.92:0/1247341257 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88e8100450 0x7f88e81994e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 >> 172.21.15.92:0/1247341257 conn(0x7f88e80fc2c0 msgr2=0x7f88e80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 shutdown_connections 2024-09-12T02:10:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:09.955+0000 7f88ed346640 1 -- 172.21.15.92:0/1247341257 wait complete. 2024-09-12T02:10:10.555 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:10 smithi003 ceph-mon[31476]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:10.556 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:10 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/1247341257' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:11.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:11 smithi092 systemd[1]: Started Ceph mon.c for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:10:11.620 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:10:11.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:10:11.877 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:10:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: mon.a calling monitor election 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: mon.c calling monitor election 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: monmap epoch 2 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: last_changed 2024-09-12T02:10:11.100526+0000 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: min_mon_release 19 (squid) 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: election_strategy: 1 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: fsmap 2024-09-12T02:10:16.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: mgrmap e13: x(active, since 39s) 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: overall HEALTH_OK 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:16.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: mon.a calling monitor election 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: mon.c calling monitor election 2024-09-12T02:10:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: monmap epoch 2 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: last_changed 2024-09-12T02:10:11.100526+0000 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: min_mon_release 19 (squid) 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: election_strategy: 1 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: fsmap 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: mgrmap e13: x(active, since 39s) 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: overall HEALTH_OK 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:16.542 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1891432043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 msgr2=0x7f10c40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1891432043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10c40059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f10dc102b10 tx=0x7f10cc02c490 comp rx=0 tx=0).stop 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1891432043 shutdown_connections 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1891432043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10c40059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1891432043 >> 172.21.15.92:0/1891432043 conn(0x7f10dc0fd380 msgr2=0x7f10dc0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1891432043 shutdown_connections 2024-09-12T02:10:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.690+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1891432043 wait complete. 2024-09-12T02:10:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.691+0000 7f10e1ae0640 1 Processor -- start 2024-09-12T02:10:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.691+0000 7f10e1ae0640 1 -- start start 2024-09-12T02:10:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10e1ae0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10dc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10e1ae0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:16.693 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10e1ae0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc1097a0 con 0x7f10c4003530 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10e1ae0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc109620 con 0x7f10dc197830 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10dc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52550/0 (socket says 172.21.15.92:52550) 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 -- 172.21.15.92:0/1465765903 learned_addr learned my addr 172.21.15.92:0/1465765903 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 -- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 msgr2=0x7f10dc19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 -- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 msgr2=0x7f10dc19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10daffd640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10db7fe640 1 -- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 msgr2=0x7f10dc19bbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10db7fe640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:16.694 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.692+0000 7f10db7fe640 1 -- 172.21.15.92:0/1465765903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc197e50 con 0x7f10c4003530 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10db7fe640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10dc1972f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10cc005f30 tx=0x7f10cc033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f10cc038040 con 0x7f10c4003530 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10cc032e00 con 0x7f10c4003530 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc19c100 con 0x7f10c4003530 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f10cc004a00 con 0x7f10c4003530 2024-09-12T02:10:16.695 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.693+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc19c640 con 0x7f10c4003530 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.694+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f10cc043020 con 0x7f10c4003530 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.694+0000 7f10d8ff9640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 0x7f10c803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.694+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc102dc0 con 0x7f10c4003530 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.694+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f10cc06e5c0 con 0x7f10c4003530 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.694+0000 7f10daffd640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 0x7f10c803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:16.696 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.695+0000 7f10daffd640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 0x7f10c803b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009100 tx=0x7f10d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:16.700 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.700+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10cc04a020 con 0x7f10c4003530 2024-09-12T02:10:16.938 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.938+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f10dc190f80 con 0x7f10c4003530 2024-09-12T02:10:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.939+0000 7f10d8ff9640 1 -- 172.21.15.92:0/1465765903 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1007 (secure 0 0 0) 0x7f10cc04cbc0 con 0x7f10c4003530 2024-09-12T02:10:16.939 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:10:16.939 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","modified":"2024-09-12T02:10:11.100526Z","created":"2024-09-12T02:08:36.454578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:10:16.939 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 2 2024-09-12T02:10:16.941 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 msgr2=0x7f10c803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 0x7f10c803b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009100 tx=0x7f10d0008040 comp rx=0 tx=0).stop 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 msgr2=0x7f10dc1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10dc1972f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10cc005f30 tx=0x7f10cc033680 comp rx=0 tx=0).stop 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 shutdown_connections 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10c8038e50 0x7f10c803b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10dc197830 0x7f10dc19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 --2- 172.21.15.92:0/1465765903 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10c4003530 0x7f10dc1972f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 >> 172.21.15.92:0/1465765903 conn(0x7f10dc0fd380 msgr2=0x7f10dc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 shutdown_connections 2024-09-12T02:10:16.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:16.941+0000 7f10e1ae0640 1 -- 172.21.15.92:0/1465765903 wait complete. 2024-09-12T02:10:17.226 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:17 smithi092 ceph-mon[36089]: Deploying daemon mon.b on smithi026 2024-09-12T02:10:17.226 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:17 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/1465765903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:17.226 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:17.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:17 smithi003 ceph-mon[31476]: Deploying daemon mon.b on smithi026 2024-09-12T02:10:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:17 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/1465765903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:18.513 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:10:18.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:10:18.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:18 smithi003 ceph-mon[31476]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:18.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:18 smithi092 ceph-mon[36089]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:18.763 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:10:19.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.206+0000 7f052bbce640 1 -- 172.21.15.92:0/746979479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524107680 msgr2=0x7f0524107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:19.210 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.206+0000 7f052bbce640 1 --2- 172.21.15.92:0/746979479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524107680 0x7f0524107a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0520009880 tx=0x7f052002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- 172.21.15.92:0/746979479 shutdown_connections 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 --2- 172.21.15.92:0/746979479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524107680 0x7f0524107a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- 172.21.15.92:0/746979479 >> 172.21.15.92:0/746979479 conn(0x7f05240fd380 msgr2=0x7f05240ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- 172.21.15.92:0/746979479 shutdown_connections 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- 172.21.15.92:0/746979479 wait complete. 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 Processor -- start 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- start start 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 0x7f052419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f052410bd40 con 0x7f0524197960 2024-09-12T02:10:19.211 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.207+0000 7f052bbce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f052410bbc0 con 0x7f0524107680 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52556/0 (socket says 172.21.15.92:52556) 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f0529169640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 0x7f052419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 -- 172.21.15.92:0/841088639 learned_addr learned my addr 172.21.15.92:0/841088639 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 -- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 msgr2=0x7f0524197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 -- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 msgr2=0x7f0524197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:10:19.212 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052996a640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f0529169640 1 -- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 msgr2=0x7f0524197420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f0529169640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f0529169640 1 -- 172.21.15.92:0/841088639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0524197f80 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f0529169640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 0x7f052419bcf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f051400b3d0 tx=0x7f051400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0514011630 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0514011cb0 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0514010df0 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052419c230 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.208+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052419c770 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.209+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0514018020 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.209+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0524102dc0 con 0x7f0524197960 2024-09-12T02:10:19.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.210+0000 7f051affd640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 0x7f051003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:19.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.210+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f051404bcb0 con 0x7f0524197960 2024-09-12T02:10:19.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.210+0000 7f052996a640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 0x7f051003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:19.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.210+0000 7f052996a640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 0x7f051003b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05200096c0 tx=0x7f0520005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:19.215 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.215+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f051401f020 con 0x7f0524197960 2024-09-12T02:10:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.446+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f052419ca80 con 0x7f0524197960 2024-09-12T02:10:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.446+0000 7f051affd640 1 -- 172.21.15.92:0/841088639 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1007 (secure 0 0 0) 0x7f05140172c0 con 0x7f0524197960 2024-09-12T02:10:19.447 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:10:19.447 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":2,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","modified":"2024-09-12T02:10:11.100526Z","created":"2024-09-12T02:08:36.454578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:10:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 2 2024-09-12T02:10:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.448+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 msgr2=0x7f051003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.448+0000 7f052bbce640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 0x7f051003b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05200096c0 tx=0x7f0520005a40 comp rx=0 tx=0).stop 2024-09-12T02:10:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.448+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 msgr2=0x7f052419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.448+0000 7f052bbce640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 0x7f052419bcf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f051400b3d0 tx=0x7f051400b890 comp rx=0 tx=0).stop 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 shutdown_connections 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0510038e00 0x7f051003b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0524197960 0x7f052419bcf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 --2- 172.21.15.92:0/841088639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0524107680 0x7f0524197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 >> 172.21.15.92:0/841088639 conn(0x7f05240fd380 msgr2=0x7f05240fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 shutdown_connections 2024-09-12T02:10:19.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:19.449+0000 7f052bbce640 1 -- 172.21.15.92:0/841088639 wait complete. 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: e1 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: btime 2024-09-12T02:08:39:464980+0000 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: legacy client fscid: -1 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout: No filesystems configured 2024-09-12T02:10:19.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:10:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1504885953' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1504885953' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:10:20.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c;count:3 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Reconfiguring daemon mon.a on smithi003 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:20.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Deploying daemon mon.c on smithi092 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/1169635911' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/1247341257' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.a calling monitor election 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.c calling monitor election 2024-09-12T02:10:20.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: monmap epoch 2 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: last_changed 2024-09-12T02:10:11.100526+0000 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: min_mon_release 19 (squid) 2024-09-12T02:10:20.271 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: election_strategy: 1 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: fsmap 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mgrmap e13: x(active, since 39s) 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: overall HEALTH_OK 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: Deploying daemon mon.b on smithi026 2024-09-12T02:10:20.272 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/1465765903' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:20.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:20.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:20.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 ceph-mon[35496]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:10:20.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 bash[35477]: 0761ad51343b16b48c4c1c2936650ec79258d8c425aebfc9759395cbb969a1e8 2024-09-12T02:10:20.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:19 smithi026 systemd[1]: Started Ceph mon.b for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:10:21.042 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:10:21.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:10:21.314 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:10:24.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584140002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4003530 msgr2=0x7fc5a40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:24.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584140002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4003530 0x7fc5a40059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc1016e0 tx=0x7fc5ac02c150 comp rx=0 tx=0).stop 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584140002 shutdown_connections 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584140002 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5a4003530 0x7fc5a40059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584140002 >> 172.21.15.92:0/3584140002 conn(0x7fc5bc0fc2c0 msgr2=0x7fc5bc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584140002 shutdown_connections 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.947+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584140002 wait complete. 2024-09-12T02:10:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.948+0000 7fc5c17c6640 1 Processor -- start 2024-09-12T02:10:24.951 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.948+0000 7fc5c17c6640 1 -- start start 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 0x7fc5bc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc1086b0 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc108530 con 0x7fc5bc199af0 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5c17c6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc108830 con 0x7fc5a4003530 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41568/0 (socket says 172.21.15.92:41568) 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 -- 172.21.15.92:0/3584659611 learned_addr learned my addr 172.21.15.92:0/3584659611 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5bb7fe640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 0x7fc5bc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:24.952 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5ba7fc640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 msgr2=0x7fc5bc1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 msgr2=0x7fc5bc1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5baffd640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5ba7fc640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 msgr2=0x7fc5bc19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5ba7fc640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 msgr2=0x7fc5bc19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5ba7fc640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.949+0000 7fc5ba7fc640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 msgr2=0x7fc5bc1995b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 msgr2=0x7fc5bc19de80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:10:24.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 -- 172.21.15.92:0/3584659611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc1a6130 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5bb7fe640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 0x7fc5bc19e870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bd50 tx=0x7fc5b400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4019070 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4014920 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc1a6360 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4012990 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.950+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc1a6870 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.951+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc5b401c070 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.951+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc1097e0 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.952+0000 7fc5a3fff640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 0x7fc5a803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.952+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc5b404be20 con 0x7fc5bc19e3c0 2024-09-12T02:10:24.954 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.952+0000 7fc5baffd640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 0x7fc5a803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:24.955 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.952+0000 7fc5baffd640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 0x7fc5a803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005fd0 tx=0x7fc5ac005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:24.957 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:24.957+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5b4013ba0 con 0x7fc5bc19e3c0 2024-09-12T02:10:25.197 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.196+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc5bc1a6c60 con 0x7fc5bc19e3c0 2024-09-12T02:10:25.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:25.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: mon.a calling monitor election 2024-09-12T02:10:25.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: mon.c calling monitor election 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: monmap epoch 3 2024-09-12T02:10:25.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: min_mon_release 19 (squid) 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: election_strategy: 1 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: fsmap 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: mgrmap e13: x(active, since 48s) 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: overall HEALTH_OK 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.199 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.200 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.200 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.197+0000 7fc5a3fff640 1 -- 172.21.15.92:0/3584659611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1271 (secure 0 0 0) 0x7fc5b400e090 con 0x7fc5bc19e3c0 2024-09-12T02:10:25.200 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:10:25.200 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","modified":"2024-09-12T02:10:19.786375Z","created":"2024-09-12T02:08:36.454578Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:10:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:dumped monmap epoch 3 2024-09-12T02:10:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 msgr2=0x7fc5a803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 0x7fc5a803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac005fd0 tx=0x7fc5ac005d50 comp rx=0 tx=0).stop 2024-09-12T02:10:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 msgr2=0x7fc5bc19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:25.201 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 0x7fc5bc19e870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400bd50 tx=0x7fc5b400be40 comp rx=0 tx=0).stop 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 shutdown_connections 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc5a80394f0 0x7fc5a803b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc5bc19e3c0 0x7fc5bc19e870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5bc199af0 0x7fc5bc19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 --2- 172.21.15.92:0/3584659611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a4003530 0x7fc5bc1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 >> 172.21.15.92:0/3584659611 conn(0x7fc5bc0fc2c0 msgr2=0x7fc5bc06a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 shutdown_connections 2024-09-12T02:10:25.202 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:25.200+0000 7fc5c17c6640 1 -- 172.21.15.92:0/3584659611 wait complete. 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: mon.a calling monitor election 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: mon.c calling monitor election 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:25.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: monmap epoch 3 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:25.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: min_mon_release 19 (squid) 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: election_strategy: 1 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: fsmap 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: mgrmap e13: x(active, since 48s) 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: overall HEALTH_OK 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:25.773 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T02:10:25.773 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T02:10:26.040 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:26.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:25 smithi003 ceph-mon[31476]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:26.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:26.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:25 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3584659611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:26.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:26.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:25 smithi092 ceph-mon[36089]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:26.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:26.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:25 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3584659611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:26.196 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:26.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 -- 172.21.15.3:0/2696915479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4102910 msgr2=0x7efea4102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 --2- 172.21.15.3:0/2696915479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4102910 0x7efea4102cf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe94009880 tx=0x7efe9402a670 comp rx=0 tx=0).stop 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 -- 172.21.15.3:0/2696915479 shutdown_connections 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 --2- 172.21.15.3:0/2696915479 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4102910 0x7efea4102cf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 -- 172.21.15.3:0/2696915479 >> 172.21.15.3:0/2696915479 conn(0x7efea40fc290 msgr2=0x7efea40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 -- 172.21.15.3:0/2696915479 shutdown_connections 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.619+0000 7efea8fa7640 1 -- 172.21.15.3:0/2696915479 wait complete. 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.620+0000 7efea8fa7640 1 Processor -- start 2024-09-12T02:10:26.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.620+0000 7efea8fa7640 1 -- start start 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 0x7efea419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efea410bd90 con 0x7efea4199ad0 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efea410bc10 con 0x7efea419f910 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea8fa7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efea410bf10 con 0x7efea4102910 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53036/0 (socket says 172.21.15.3:53036) 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 0x7efea419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea2d9d640 1 -- 172.21.15.3:0/377423261 learned_addr learned my addr 172.21.15.3:0/377423261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:10:26.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea1d9b640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea1d9b640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 msgr2=0x7efea4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea1d9b640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 msgr2=0x7efea4199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea1d9b640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea1d9b640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea259c640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 msgr2=0x7efea4199590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea259c640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea259c640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 msgr2=0x7efea4199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.621+0000 7efea259c640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efea259c640 1 -- 172.21.15.3:0/377423261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea41a62a0 con 0x7efea4102910 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efea2d9d640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efea259c640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 0x7efea419f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe94005ab0 tx=0x7efe940042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:26.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9403f030 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe94003ce0 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94035070 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea41a6470 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.622+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea41a6900 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.623+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea4102d70 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.623+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7efe94042030 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.624+0000 7efe937fe640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 0x7efe8c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.624+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe9406c960 con 0x7efea4102910 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.624+0000 7efea1d9b640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 0x7efe8c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:26.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.624+0000 7efea1d9b640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 0x7efe8c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:26.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.629+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe9403ac70 con 0x7efea4102910 2024-09-12T02:10:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.895+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efea41a6c50 con 0x7efea4102910 2024-09-12T02:10:26.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.896+0000 7efe937fe640 1 -- 172.21.15.3:0/377423261 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+269 (secure 0 0 0) 0x7efe94030c30 con 0x7efea4102910 2024-09-12T02:10:26.897 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:26.897 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-09-12T02:10:26.897 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:26.897 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] 2024-09-12T02:10:26.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 msgr2=0x7efe8c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:26.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 0x7efe8c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).stop 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 msgr2=0x7efea419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 0x7efea419f3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe94005ab0 tx=0x7efe940042f0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 shutdown_connections 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe8c0394f0 0x7efe8c03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efea4199ad0 0x7efea4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea419f910 0x7efea4199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 --2- 172.21.15.3:0/377423261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efea4102910 0x7efea419f3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 >> 172.21.15.3:0/377423261 conn(0x7efea40fc290 msgr2=0x7efea4107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 shutdown_connections 2024-09-12T02:10:26.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:26.900+0000 7efea8fa7640 1 -- 172.21.15.3:0/377423261 wait complete. 2024-09-12T02:10:27.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:10:27.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: mon.a calling monitor election 2024-09-12T02:10:27.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: mon.c calling monitor election 2024-09-12T02:10:27.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: monmap epoch 3 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:27.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: min_mon_release 19 (squid) 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: election_strategy: 1 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: fsmap 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: mgrmap e13: x(active, since 48s) 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: overall HEALTH_OK 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:27.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3584659611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:10:27.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mon.b calling monitor election 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-12T02:10:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mon.b calling monitor election 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mon.c calling monitor election 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mon.a calling monitor election 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Updating smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: monmap epoch 3 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:27.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: min_mon_release 19 (squid) 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: election_strategy: 1 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: fsmap 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: mgrmap e13: x(active, since 50s) 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: overall HEALTH_OK 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/377423261' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Reconfiguring mon.a (monmap changed)... 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.531 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:27 smithi003 ceph-mon[31476]: Reconfiguring daemon mon.a on smithi003 2024-09-12T02:10:27.618 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T02:10:27.618 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:10:27.618 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:10:27.647 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:10:27.647 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:10:27.714 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:10:27.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mon.b calling monitor election 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mon.b calling monitor election 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mon.c calling monitor election 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mon.a calling monitor election 2024-09-12T02:10:27.738 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Updating smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: monmap epoch 3 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: min_mon_release 19 (squid) 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: election_strategy: 1 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: fsmap 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:27.739 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: mgrmap e13: x(active, since 50s) 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: overall HEALTH_OK 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/377423261' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Reconfiguring mon.a (monmap changed)... 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:27.740 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.741 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:27 smithi026 ceph-mon[35496]: Reconfiguring daemon mon.a on smithi003 2024-09-12T02:10:27.742 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:10:27.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mon.b calling monitor election 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi003:/etc/ceph/ceph.conf 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi026:/etc/ceph/ceph.conf 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi092:/etc/ceph/ceph.conf 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mon.b calling monitor election 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mon.c calling monitor election 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mon.a calling monitor election 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Updating smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: monmap epoch 3 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: last_changed 2024-09-12T02:10:19.786375+0000 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: created 2024-09-12T02:08:36.454578+0000 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: min_mon_release 19 (squid) 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: election_strategy: 1 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-09-12T02:10:27.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: 1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.c 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: 2: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.b 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: fsmap 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: mgrmap e13: x(active, since 50s) 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: overall HEALTH_OK 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/377423261' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:27.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Reconfiguring mon.a (monmap changed)... 2024-09-12T02:10:27.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:27.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:27.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:27.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:27 smithi092 ceph-mon[36089]: Reconfiguring daemon mon.a on smithi003 2024-09-12T02:10:27.806 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:10:27.806 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:10:27.834 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:10:27.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:10:27.900 INFO:tasks.cephadm:Adding mgr.x on smithi003 2024-09-12T02:10:27.900 INFO:tasks.cephadm:Adding mgr.y on smithi026 2024-09-12T02:10:27.900 INFO:tasks.cephadm:Adding mgr.z on smithi092 2024-09-12T02:10:27.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi003=x;smithi026=y;smithi092=z' 2024-09-12T02:10:28.175 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:10:28.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:28 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:28.587 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:28 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.610+0000 7f33a6d46640 1 -- 172.21.15.92:0/1355521566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0101bf0 msgr2=0x7f33a0101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:28.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.610+0000 7f33a6d46640 1 --2- 172.21.15.92:0/1355521566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0101bf0 0x7f33a0101ff0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3394009880 tx=0x7f339402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- 172.21.15.92:0/1355521566 shutdown_connections 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 --2- 172.21.15.92:0/1355521566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0101bf0 0x7f33a0101ff0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- 172.21.15.92:0/1355521566 >> 172.21.15.92:0/1355521566 conn(0x7f33a00fb540 msgr2=0x7f33a00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- 172.21.15.92:0/1355521566 shutdown_connections 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- 172.21.15.92:0/1355521566 wait complete. 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 Processor -- start 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- start start 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0197560 0x7f33a019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 0x7f33a019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01066f0 con 0x7f33a0197560 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0106570 con 0x7f33a01979e0 2024-09-12T02:10:28.613 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a6d46640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0106870 con 0x7f33a0101bf0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a4ae2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a4ae2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41584/0 (socket says 172.21.15.92:41584) 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f33a4ae2640 1 -- 172.21.15.92:0/88629848 learned_addr learned my addr 172.21.15.92:0/88629848 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.611+0000 7f339ffff640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0197560 0x7f33a019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a52e3640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 0x7f33a019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 -- 172.21.15.92:0/88629848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 msgr2=0x7f33a019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 0x7f33a019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 -- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0197560 msgr2=0x7f33a019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0197560 0x7f33a019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 -- 172.21.15.92:0/88629848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a019f420 con 0x7f33a0101bf0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a52e3640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 0x7f33a019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a4ae2640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3394031610 tx=0x7f33940043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:28.614 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339403f030 con 0x7f33a0101bf0 2024-09-12T02:10:28.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a019f6b0 con 0x7f33a0101bf0 2024-09-12T02:10:28.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3394003ce0 con 0x7f33a0101bf0 2024-09-12T02:10:28.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394035070 con 0x7f33a0101bf0 2024-09-12T02:10:28.615 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.612+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a019fc10 con 0x7f33a0101bf0 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.613+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f3394042030 con 0x7f33a0101bf0 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.613+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a0190880 con 0x7f33a0101bf0 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.613+0000 7f339dffb640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 0x7f339003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.614+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f339406cbc0 con 0x7f33a0101bf0 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.614+0000 7f339ffff640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 0x7f339003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:28.616 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.614+0000 7f339ffff640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 0x7f339003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33a0198540 tx=0x7f338c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:28.618 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.618+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f339403ac70 con 0x7f33a0101bf0 2024-09-12T02:10:28.798 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.797+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi026=y;smithi092=z", "target": ["mon-mgr", ""]}) -- 0x7f33a00626c0 con 0x7f33900394f0 2024-09-12T02:10:28.803 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.803+0000 7f339dffb640 1 -- 172.21.15.92:0/88629848 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f33a00626c0 con 0x7f33900394f0 2024-09-12T02:10:28.803 INFO:teuthology.orchestra.run.smithi092.stdout:Scheduled mgr update... 2024-09-12T02:10:28.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 msgr2=0x7f339003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:28.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 0x7f339003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33a0198540 tx=0x7f338c009190 comp rx=0 tx=0).stop 2024-09-12T02:10:28.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 msgr2=0x7f33a0197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:28.805 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3394031610 tx=0x7f33940043c0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 shutdown_connections 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f33900394f0 0x7f339003b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33a01979e0 0x7f33a019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33a0197560 0x7f33a019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 --2- 172.21.15.92:0/88629848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33a0101bf0 0x7f33a0197020 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 >> 172.21.15.92:0/88629848 conn(0x7f33a00fb540 msgr2=0x7f33a00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 shutdown_connections 2024-09-12T02:10:28.806 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:10:28.805+0000 7f33a6d46640 1 -- 172.21.15.92:0/88629848 wait complete. 2024-09-12T02:10:28.978 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:28 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:10:29.435 DEBUG:teuthology.orchestra.run.smithi026:mgr.y> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T02:10:29.438 DEBUG:teuthology.orchestra.run.smithi092:mgr.z> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T02:10:29.441 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T02:10:29.441 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:10:29.441 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:10:29.462 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:10:29.462 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-09-12T02:10:29.521 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 02:09:46.930146252 +0000 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 02:04:50.212941845 +0000 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 02:04:50.212941845 +0000 2024-09-12T02:10:29.522 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-12 02:04:50.212941845 +0000 2024-09-12T02:10:29.522 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:10:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T02:10:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T02:10:29.592 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000227294 s, 2.3 MB/s 2024-09-12T02:10:29.594 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:10:29.653 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 02:09:46.965145815 +0000 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 02:04:50.544937567 +0000 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 02:04:50.544937567 +0000 2024-09-12T02:10:29.711 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-12 02:04:50.544937567 +0000 2024-09-12T02:10:29.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:10:29.779 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T02:10:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T02:10:29.780 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000213692 s, 2.4 MB/s 2024-09-12T02:10:29.781 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:10:29.841 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-09-12T02:10:29.855 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:29.855 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.855 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.855 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: Reconfiguring daemon mon.b on smithi026 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi026=y;smithi092=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: Saving service mgr spec with placement smithi003=x;smithi026=y;smithi092=z;count:3 2024-09-12T02:10:29.856 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:29 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 02:09:47.001145365 +0000 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 02:04:50.871933353 +0000 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 02:04:50.871933353 +0000 2024-09-12T02:10:29.900 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-12 02:04:50.871933353 +0000 2024-09-12T02:10:29.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:10:29.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:29.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:10:29.964 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: Reconfiguring daemon mon.b on smithi026 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi026=y;smithi092=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: Saving service mgr spec with placement smithi003=x;smithi026=y;smithi092=z;count:3 2024-09-12T02:10:29.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:29 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:29.967 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T02:10:29.967 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T02:10:29.967 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000195121 s, 2.6 MB/s 2024-09-12T02:10:29.969 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:10:30.027 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-09-12T02:10:30.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: Reconfiguring daemon mon.b on smithi026 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi026=y;smithi092=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: Saving service mgr spec with placement smithi003=x;smithi026=y;smithi092=z;count:3 2024-09-12T02:10:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:29 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:30.083 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 02:09:47.036144928 +0000 2024-09-12T02:10:30.084 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 02:04:51.185929306 +0000 2024-09-12T02:10:30.084 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 02:04:51.185929306 +0000 2024-09-12T02:10:30.084 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-12 02:04:51.185929306 +0000 2024-09-12T02:10:30.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:10:30.149 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-09-12T02:10:30.149 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-09-12T02:10:30.149 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000205861 s, 2.5 MB/s 2024-09-12T02:10:30.150 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:10:30.207 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-12T02:10:30.207 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:10:30.227 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:10:30.227 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-09-12T02:10:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:10:30.284 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:30.284 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T02:10:30.284 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:30.285 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:30.285 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-12 02:10:19.234932093 +0000 2024-09-12T02:10:30.285 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-12 02:05:07.771696905 +0000 2024-09-12T02:10:30.285 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-12 02:05:07.771696905 +0000 2024-09-12T02:10:30.285 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-12 02:05:07.771696905 +0000 2024-09-12T02:10:30.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:10:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-12T02:10:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-12T02:10:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000122677 s, 4.2 MB/s 2024-09-12T02:10:30.353 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:10:30.411 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:30.467 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-12 02:10:19.235932078 +0000 2024-09-12T02:10:30.468 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-12 02:05:08.095691801 +0000 2024-09-12T02:10:30.468 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-12 02:05:08.095691801 +0000 2024-09-12T02:10:30.468 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-12 02:05:08.095691801 +0000 2024-09-12T02:10:30.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:10:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-12T02:10:30.533 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-12T02:10:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000118609 s, 4.3 MB/s 2024-09-12T02:10:30.535 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:10:30.592 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-12 02:10:19.235932078 +0000 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-12 02:05:08.423686634 +0000 2024-09-12T02:10:30.648 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-12 02:05:08.423686634 +0000 2024-09-12T02:10:30.649 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-12 02:05:08.423686634 +0000 2024-09-12T02:10:30.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:10:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-12T02:10:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-12T02:10:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000106187 s, 4.8 MB/s 2024-09-12T02:10:30.713 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:10:30.770 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-09-12T02:10:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:10:30.826 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:30.826 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-12 02:10:19.235932078 +0000 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-12 02:05:08.743681592 +0000 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-12 02:05:08.743681592 +0000 2024-09-12T02:10:30.827 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-09-12 02:05:08.743681592 +0000 2024-09-12T02:10:30.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:10:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-12T02:10:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-12T02:10:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000110191 s, 4.6 MB/s 2024-09-12T02:10:30.892 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:10:30.948 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-09-12T02:10:30.948 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:10:30.964 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:10:30.965 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-09-12T02:10:31.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:30 smithi026 ceph-mon[35496]: Reconfiguring daemon mon.c on smithi092 2024-09-12T02:10:31.020 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-12 02:10:10.020094319 +0000 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-12 02:05:11.511755558 +0000 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-12 02:05:11.511755558 +0000 2024-09-12T02:10:31.021 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-12 02:05:11.511755558 +0000 2024-09-12T02:10:31.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:10:31.085 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-12T02:10:31.086 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-12T02:10:31.086 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000250609 s, 2.0 MB/s 2024-09-12T02:10:31.087 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:10:31.143 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-09-12T02:10:31.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:10:31.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:31.197 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:31.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:31.198 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:30 smithi092 ceph-mon[36089]: Reconfiguring daemon mon.c on smithi092 2024-09-12T02:10:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:10:31.199 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-09-12T02:10:31.199 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-12 02:10:10.020094319 +0000 2024-09-12T02:10:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-12 02:05:11.841745384 +0000 2024-09-12T02:10:31.200 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-12 02:05:11.841745384 +0000 2024-09-12T02:10:31.200 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-12 02:05:11.841745384 +0000 2024-09-12T02:10:31.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:10:31.264 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-12T02:10:31.264 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-12T02:10:31.264 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000172005 s, 3.0 MB/s 2024-09-12T02:10:31.266 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:10:31.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:31.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:10:31.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:10:31.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:10:31.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:31.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:30 smithi003 ceph-mon[31476]: Reconfiguring daemon mon.c on smithi092 2024-09-12T02:10:31.321 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-12 02:10:10.021094289 +0000 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-12 02:05:12.164735426 +0000 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-12 02:05:12.164735426 +0000 2024-09-12T02:10:31.379 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-12 02:05:12.164735426 +0000 2024-09-12T02:10:31.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:10:31.443 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-12T02:10:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-12T02:10:31.444 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000263193 s, 1.9 MB/s 2024-09-12T02:10:31.445 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:10:31.501 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-09-12T02:10:31.557 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:10:31.557 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-09-12 02:10:10.021094289 +0000 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-09-12 02:05:12.470725993 +0000 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-09-12 02:05:12.470725993 +0000 2024-09-12T02:10:31.558 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-09-12 02:05:12.470725993 +0000 2024-09-12T02:10:31.558 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:10:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-09-12T02:10:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-09-12T02:10:31.622 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000214623 s, 2.4 MB/s 2024-09-12T02:10:31.623 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:10:31.678 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2024-09-12T02:10:31.679 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:10:31.939 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:32.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:32.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:32.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.355 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:32 smithi003 ceph-mon[31476]: Deploying daemon mgr.z on smithi092 2024-09-12T02:10:32.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:32 smithi026 ceph-mon[35496]: Deploying daemon mgr.z on smithi092 2024-09-12T02:10:32.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:32 smithi092 ceph-mon[36089]: Deploying daemon mgr.z on smithi092 2024-09-12T02:10:33.698 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:33 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:33.551+0000 7fe52e9e9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:10:33.698 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:33 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:33.697+0000 7fe52e9e9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:10:34.038 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:33 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:33.762+0000 7fe52e9e9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:10:34.039 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:33 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:33.906+0000 7fe52e9e9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:10:34.201 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:10:34.228 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2024-09-12T02:10:34.357 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:34.044+0000 7fe52e9e9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:10:34.357 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:34.356+0000 7fe52e9e9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:10:34.357 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:34.358 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:34.481 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:10:34.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:34.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:34.788 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:34.416+0000 7fe52e9e9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:10:34.789 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:34.598+0000 7fe52e9e9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:10:34.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 -- 172.21.15.3:0/564038941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f81026a0 msgr2=0x7f86f8104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 --2- 172.21.15.3:0/564038941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f81026a0 0x7f86f8104a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a410 comp rx=0 tx=0).stop 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 -- 172.21.15.3:0/564038941 shutdown_connections 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 --2- 172.21.15.3:0/564038941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f8107900 0x7f86f8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 --2- 172.21.15.3:0/564038941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 --2- 172.21.15.3:0/564038941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f81026a0 0x7f86f8104a90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 -- 172.21.15.3:0/564038941 >> 172.21.15.3:0/564038941 conn(0x7f86f80fc2a0 msgr2=0x7f86f80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 -- 172.21.15.3:0/564038941 shutdown_connections 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.911+0000 7f870039f640 1 -- 172.21.15.3:0/564038941 wait complete. 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.912+0000 7f870039f640 1 Processor -- start 2024-09-12T02:10:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.912+0000 7f870039f640 1 -- start start 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f81026a0 0x7f86f8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d8d0 con 0x7f86f8107900 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810d750 con 0x7f86f81026a0 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f870039f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f86f810da50 con 0x7f86f8104fd0 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe13b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f81026a0 0x7f86f8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41930/0 (socket says 172.21.15.3:41930) 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 -- 172.21.15.3:0/453588082 learned_addr learned my addr 172.21.15.3:0/453588082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:10:34.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fd93a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 -- 172.21.15.3:0/453588082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 msgr2=0x7f86f8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 -- 172.21.15.3:0/453588082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f81026a0 msgr2=0x7f86f8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f81026a0 0x7f86f8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 -- 172.21.15.3:0/453588082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f819fb20 con 0x7f86f8107900 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fd93a640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86fe93c640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86f000cce0 tx=0x7f86f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.913+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000f450 con 0x7f86f8107900 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.914+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f000fad0 con 0x7f86f8107900 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.914+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f819fe10 con 0x7f86f8107900 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.914+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018c10 con 0x7f86f8107900 2024-09-12T02:10:34.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.914+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f81a02c0 con 0x7f86f8107900 2024-09-12T02:10:34.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.914+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f86f00213f0 con 0x7f86f8107900 2024-09-12T02:10:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.915+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f80fdd90 con 0x7f86f8107900 2024-09-12T02:10:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.915+0000 7f86ef7fe640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 0x7f86e403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:10:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.915+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f86f004c8e0 con 0x7f86f8107900 2024-09-12T02:10:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.915+0000 7f86fe13b640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 0x7f86e403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:10:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.915+0000 7f86fe13b640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 0x7f86e403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86f40314e0 tx=0x7f86f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:10:34.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:34.920+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86f0014030 con 0x7f86f8107900 2024-09-12T02:10:35.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:35.105+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f86f8062630 con 0x7f86e40392c0 2024-09-12T02:10:35.488 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-mon[35496]: Deploying daemon mgr.y on smithi026 2024-09-12T02:10:35.489 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:10:35.489 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:10:35.489 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:35.489 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 systemd[1]: Starting Ceph mgr.y for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:10:35.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:35 smithi003 ceph-mon[31476]: Deploying daemon mgr.y on smithi026 2024-09-12T02:10:35.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:10:35.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:10:35.521 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:35.538 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:35.473+0000 7fe52e9e9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:10:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-mon[36089]: Deploying daemon mgr.y on smithi026 2024-09-12T02:10:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:10:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:10:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:35.765 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 podman[36850]: 2024-09-12 02:10:35.488562317 +0000 UTC m=+0.100881393 container create 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T02:10:35.765 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 podman[36850]: 2024-09-12 02:10:35.404359052 +0000 UTC m=+0.016678078 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:10:35.765 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 podman[36850]: 2024-09-12 02:10:35.699400016 +0000 UTC m=+0.311719035 container init 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T02:10:35.765 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 podman[36850]: 2024-09-12 02:10:35.702272146 +0000 UTC m=+0.314591168 container start 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T02:10:35.765 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-mgr[36871]: -- 172.21.15.26:0/1551877087 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa06e690e0 con 0x55aa06e47000 2024-09-12T02:10:36.038 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:35 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:35.540+0000 7fe52e9e9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:10:36.044 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 bash[36850]: 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 2024-09-12T02:10:36.045 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 systemd[1]: Started Ceph mgr.y for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:10:36.045 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:35.825+0000 7fc3090ce040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:10:36.045 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:35 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:35.886+0000 7fc3090ce040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:10:36.045 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.044+0000 7fc3090ce040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:10:36.437 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.114+0000 7fc3090ce040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:10:36.437 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.280+0000 7fc3090ce040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:10:36.437 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.437+0000 7fc3090ce040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:36.538 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.277+0000 7fe52e9e9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.404+0000 7fe52e9e9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:36.836 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.546+0000 7fe52e9e9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:10:36.836 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.610+0000 7fe52e9e9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:10:36.837 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.768+0000 7fe52e9e9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:10:36.837 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.836+0000 7fe52e9e9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:10:37.037 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.772+0000 7fc3090ce040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:10:37.038 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:36.836+0000 7fc3090ce040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:10:37.038 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:37.037+0000 7fc3090ce040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:10:37.132 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:36.896+0000 7fe52e9e9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:10:37.132 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.058+0000 7fe52e9e9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:10:37.390 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.132+0000 7fe52e9e9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:10:37.390 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.201+0000 7fe52e9e9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:10:37.390 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.287+0000 7fe52e9e9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:10:37.391 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.346+0000 7fe52e9e9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:10:37.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:37.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:37 smithi003 ceph-mon[31476]: Reconfiguring daemon mgr.x on smithi003 2024-09-12T02:10:37.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:37.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:37.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-mon[35496]: Reconfiguring daemon mgr.x on smithi003 2024-09-12T02:10:37.788 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:37.406+0000 7fe52e9e9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:37.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:10:37.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:10:37.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:10:37.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:37.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:37 smithi092 ceph-mon[36089]: Reconfiguring daemon mgr.x on smithi003 2024-09-12T02:10:38.264 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:37 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:37.959+0000 7fc3090ce040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:10:38.265 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:38.029+0000 7fc3090ce040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:10:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:38.501+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f86f000e450 con 0x7f86f8107900 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z[37892]: 2024-09-12T02:10:38.189+0000 7fe52e9e9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]: dispatch 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]': finished 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/296110041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: Standby manager daemon z started 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:10:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:10:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:38 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:38.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]': finished 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/296110041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: Standby manager daemon z started 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:10:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:38.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1064336552' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d0aab17-f2a0-467b-9ab2-ddd8fc92d297"}]': finished 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/296110041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: Standby manager daemon z started 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:10:38.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:38 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.92:0/1920483450' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:39.053 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:38.783+0000 7fc3090ce040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:10:39.053 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:38.904+0000 7fc3090ce040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:10:39.342 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.052+0000 7fc3090ce040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:10:39.342 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.116+0000 7fc3090ce040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:10:39.342 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.271+0000 7fc3090ce040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:10:39.342 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.342+0000 7fc3090ce040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:10:39.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-mon[35496]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:39.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-mon[35496]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T02:10:39.637 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:10:39.638 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.404+0000 7fc3090ce040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:10:39.638 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.569+0000 7fc3090ce040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:10:39.638 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.637+0000 7fc3090ce040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:10:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:39 smithi003 ceph-mon[31476]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:39 smithi003 ceph-mon[31476]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T02:10:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:10:39.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:39 smithi092 ceph-mon[36089]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:39.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:39 smithi092 ceph-mon[36089]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T02:10:39.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:10:39.915 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.706+0000 7fc3090ce040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:10:39.915 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.792+0000 7fc3090ce040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:10:39.915 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.853+0000 7fc3090ce040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:10:39.915 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:39.915+0000 7fc3090ce040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:10:41.014 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:10:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y[36867]: 2024-09-12T02:10:40.739+0000 7fc3090ce040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:10:41.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:41.514+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f86f001f300 con 0x7f86f8107900 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: Standby manager daemon y started 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:10:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:41 smithi026 ceph-mon[35496]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: Standby manager daemon y started 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:10:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:41 smithi003 ceph-mon[31476]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: Standby manager daemon y started 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:10:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:41 smithi092 ceph-mon[36089]: from='mgr.? 172.21.15.26:0/3760780955' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:10:42.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:42 smithi026 ceph-mon[35496]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:10:42.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:10:42.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:42 smithi003 ceph-mon[31476]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:10:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:10:42.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:42 smithi092 ceph-mon[36089]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-12T02:10:42.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:10:43.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:43 smithi003 ceph-mon[31476]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:43.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:43 smithi092 ceph-mon[36089]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:44.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:43 smithi026 ceph-mon[35496]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:45.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:45.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:45.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:45.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:45.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:46.145 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:45 smithi003 ceph-mon[31476]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:46.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:45 smithi026 ceph-mon[35496]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:46.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:45 smithi092 ceph-mon[36089]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:47.134 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:46 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:10:47.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:46 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:47.135 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:46 smithi003 ceph-mon[31476]: Deploying daemon osd.0 on smithi003 2024-09-12T02:10:47.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:46 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:10:47.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:46 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:47.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:46 smithi026 ceph-mon[35496]: Deploying daemon osd.0 on smithi003 2024-09-12T02:10:47.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:46 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:10:47.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:46 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:47.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:46 smithi092 ceph-mon[36089]: Deploying daemon osd.0 on smithi003 2024-09-12T02:10:48.207 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:47 smithi003 ceph-mon[31476]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:47 smithi026 ceph-mon[35496]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:48.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:47 smithi092 ceph-mon[36089]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:49 smithi026 ceph-mon[35496]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:49 smithi003 ceph-mon[31476]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:50.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:49 smithi092 ceph-mon[36089]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:52.227 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:51 smithi003 ceph-mon[31476]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:52.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:51 smithi026 ceph-mon[35496]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:51 smithi092 ceph-mon[36089]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:53.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:53.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:53.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:53.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:53.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:53.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:53.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:53.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:53.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:54.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:54 smithi003 ceph-mon[31476]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:54.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:54 smithi092 ceph-mon[36089]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:54 smithi026 ceph-mon[35496]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:55.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:10:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.372+0000 7f86ef7fe640 1 -- 172.21.15.3:0/453588082 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f86f8062630 con 0x7f86e40392c0 2024-09-12T02:10:56.373 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-09-12T02:10:56.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 msgr2=0x7f86e403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 0x7f86e403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86f40314e0 tx=0x7f86f4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 msgr2=0x7f86f819f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86f000cce0 tx=0x7f86f000a620 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 shutdown_connections 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f86e40392c0 0x7f86e403b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f86f8107900 0x7f86f819f4b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86f8104fd0 0x7f86f8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 --2- 172.21.15.3:0/453588082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86f81026a0 0x7f86f8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:10:56.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.375+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 >> 172.21.15.3:0/453588082 conn(0x7f86f80fc2a0 msgr2=0x7f86f8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:10:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.376+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 shutdown_connections 2024-09-12T02:10:56.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:10:56.376+0000 7f870039f640 1 -- 172.21.15.3:0/453588082 wait complete. 2024-09-12T02:10:56.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:56.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:10:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:10:56.983 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T02:10:56.986 INFO:tasks.cephadm:Deploying osd.1 on smithi026 with /dev/vg_nvme/lv_4... 2024-09-12T02:10:56.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:10:57.230 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:10:57.654 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:57 smithi026 ceph-mon[35496]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:57.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:57 smithi003 ceph-mon[31476]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:57.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:57 smithi092 ceph-mon[36089]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:58.278 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:10:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:10:58.106+0000 7f88007bc640 -1 osd.0 0 log_to_monitors true 2024-09-12T02:10:58.452 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:58 smithi026 ceph-mon[35496]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:10:58.582 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:58 smithi003 ceph-mon[31476]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:10:58.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:58 smithi092 ceph-mon[36089]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:10:59.516 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:10:59.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2024-09-12T02:10:59.754 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-mon[31476]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:59.754 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-mon[31476]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:10:59.755 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-mon[31476]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:10:59.755 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-mon[31476]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:10:59.755 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:59.755 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:10:59.401+0000 7f87fc73f640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T02:10:59.755 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:10:59 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:10:59.404+0000 7f87f8567640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:10:59.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:59 smithi026 ceph-mon[35496]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:59.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:59 smithi026 ceph-mon[35496]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:10:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:59 smithi026 ceph-mon[35496]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:10:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:59 smithi026 ceph-mon[35496]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:10:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:10:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:59.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:59 smithi092 ceph-mon[36089]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:10:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:59 smithi092 ceph-mon[36089]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:10:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:59 smithi092 ceph-mon[36089]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:10:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:59 smithi092 ceph-mon[36089]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:10:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:10:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:10:59.797 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1177628107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 msgr2=0x7fbc2c108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1177628107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c108f50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ca0 tx=0x7fbc2402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1177628107 shutdown_connections 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1177628107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c108f50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1177628107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c101d00 0x7fbc2c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1177628107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c100b00 0x7fbc2c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1177628107 >> 172.21.15.26:0/1177628107 conn(0x7fbc2c0fc290 msgr2=0x7fbc2c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.243+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1177628107 shutdown_connections 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1177628107 wait complete. 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 Processor -- start 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 -- start start 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c100b00 0x7fbc2c19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.244+0000 7fbc30dd9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc30dd9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10c760 con 0x7fbc2c101d00 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc30dd9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10c5e0 con 0x7fbc2c1026c0 2024-09-12T02:11:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc30dd9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c10c8e0 con 0x7fbc2c100b00 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc2ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49454/0 (socket says 172.21.15.26:49454) 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.26:34920/0 (socket says 172.21.15.26:34920) 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc2ad9d640 1 -- 172.21.15.26:0/1416499208 learned_addr learned my addr 172.21.15.26:0/1416499208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc2a59c640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c100b00 0x7fbc2c19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 -- 172.21.15.26:0/1416499208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c100b00 msgr2=0x7fbc2c19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c100b00 0x7fbc2c19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 -- 172.21.15.26:0/1416499208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 msgr2=0x7fbc2c1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 -- 172.21.15.26:0/1416499208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1a1cf0 con 0x7fbc2c101d00 2024-09-12T02:11:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc2ad9d640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc29d9b640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400d6d0 tx=0x7fbc1400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14017070 con 0x7fbc2c101d00 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc14009820 con 0x7fbc2c101d00 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1a1fe0 con 0x7fbc2c101d00 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.245+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14004d00 con 0x7fbc2c101d00 2024-09-12T02:11:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.246+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1a24e0 con 0x7fbc2c101d00 2024-09-12T02:11:00.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.247+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c1a2710 con 0x7fbc2c101d00 2024-09-12T02:11:00.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.247+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fbc14005220 con 0x7fbc2c101d00 2024-09-12T02:11:00.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.248+0000 7fbc237fe640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 0x7fbc180a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:00.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.248+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fbc140cceb0 con 0x7fbc2c101d00 2024-09-12T02:11:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.248+0000 7fbc2a59c640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 0x7fbc180a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.249+0000 7fbc2a59c640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 0x7fbc180a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0095f0 tx=0x7fbc1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:00.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.253+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc1409b330 con 0x7fbc2c101d00 2024-09-12T02:11:00.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:00.437+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbc2c0fe4e0 con 0x7fbc180a2ba0 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi003 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.438 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:00.439 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:00.439 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi003 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-12T02:11:00.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:00.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:00.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi003 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi003 to 17092M 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:00.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:00.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] boot 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] boot 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: osd.0 [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] boot 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: from='client.14259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:03.135 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:02 smithi026 ceph-mon[35496]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:11:03.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:02 smithi003 ceph-mon[31476]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:11:03.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:02 smithi092 ceph-mon[36089]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:11:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:04.382 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/1604558453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.383 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.383 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]': finished 2024-09-12T02:11:04.383 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:11:04.383 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:04.383 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:03 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/4204708070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/1604558453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]': finished 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:03 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/4204708070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/1604558453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.384 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]: dispatch 2024-09-12T02:11:04.385 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dcaeda7b-e940-4d6e-84b7-4b839bed3275"}]': finished 2024-09-12T02:11:04.385 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:11:04.385 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:04.385 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:03 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/4204708070' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:06.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:05 smithi026 ceph-mon[35496]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:05 smithi003 ceph-mon[31476]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:06.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:05 smithi092 ceph-mon[36089]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:07.905 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:07 smithi026 ceph-mon[35496]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:08.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:07 smithi003 ceph-mon[31476]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:08.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:07 smithi092 ceph-mon[36089]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:10.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:09 smithi026 ceph-mon[35496]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:10.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:09 smithi003 ceph-mon[31476]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:10.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:09 smithi092 ceph-mon[36089]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:11.095 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:10 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:11:11.095 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:10 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:11.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:10 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:11:11.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:10 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:11.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:10 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:11:11.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:10 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:12.084 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:11 smithi026 ceph-mon[35496]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:12.084 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:11 smithi026 ceph-mon[35496]: Deploying daemon osd.1 on smithi026 2024-09-12T02:11:12.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:11 smithi003 ceph-mon[31476]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:12.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:11 smithi003 ceph-mon[31476]: Deploying daemon osd.1 on smithi026 2024-09-12T02:11:12.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:11 smithi092 ceph-mon[36089]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:12.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:11 smithi092 ceph-mon[36089]: Deploying daemon osd.1 on smithi026 2024-09-12T02:11:14.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:13 smithi026 ceph-mon[35496]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:14.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:13 smithi003 ceph-mon[31476]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:14.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:13 smithi092 ceph-mon[36089]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:15.970 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:15 smithi026 ceph-mon[35496]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:16.091 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:15 smithi092 ceph-mon[36089]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:16.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:15 smithi003 ceph-mon[31476]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:17.167 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:17.168 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:17.168 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:17.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:17.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:17.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:17.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:17.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:17.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:18.232 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:17 smithi026 ceph-mon[35496]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:18.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:17 smithi003 ceph-mon[31476]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:18.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:17 smithi092 ceph-mon[36089]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:20.354 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:20 smithi026 ceph-mon[35496]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:20.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:20 smithi003 ceph-mon[31476]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:20.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:20 smithi092 ceph-mon[36089]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:20.626 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 1 on host 'smithi026' 2024-09-12T02:11:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.624+0000 7fbc237fe640 1 -- 172.21.15.26:0/1416499208 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc2c0fe4e0 con 0x7fbc180a2ba0 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 msgr2=0x7fbc180a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 0x7fbc180a5060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0095f0 tx=0x7fbc1c005f90 comp rx=0 tx=0).stop 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 msgr2=0x7fbc2c19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400d6d0 tx=0x7fbc1400db90 comp rx=0 tx=0).stop 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 shutdown_connections 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbc180a2ba0 0x7fbc180a5060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc2c1026c0 0x7fbc2c1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbc2c101d00 0x7fbc2c19a600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 --2- 172.21.15.26:0/1416499208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2c100b00 0x7fbc2c19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.627+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 >> 172.21.15.26:0/1416499208 conn(0x7fbc2c0fc290 msgr2=0x7fbc2c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.628+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 shutdown_connections 2024-09-12T02:11:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:11:20.628+0000 7fbc30dd9640 1 -- 172.21.15.26:0/1416499208 wait complete. 2024-09-12T02:11:21.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:21.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.323 DEBUG:teuthology.orchestra.run.smithi026:osd.1> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T02:11:21.328 INFO:tasks.cephadm:Deploying osd.2 on smithi092 with /dev/vg_nvme/lv_4... 2024-09-12T02:11:21.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:11:21.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:21.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:21.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:21.593 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:11:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:22 smithi092 ceph-mon[36089]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:22 smithi026 ceph-mon[35496]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:22.515 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:22 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:11:22.371+0000 7f81fabae640 -1 osd.1 0 log_to_monitors true 2024-09-12T02:11:22.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:22 smithi003 ceph-mon[31476]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:23.465 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:23 smithi092 ceph-mon[36089]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.466 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:23 smithi092 ceph-mon[36089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:23 smithi026 ceph-mon[35496]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:23 smithi026 ceph-mon[35496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:23 smithi003 ceph-mon[31476]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:23 smithi003 ceph-mon[31476]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:11:23.981 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:11:24.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-09-12T02:11:24.257 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.457 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:24.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:24.458 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:24.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:24.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 -- 172.21.15.92:0/2328156029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d0101c10 msgr2=0x7f32d0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 --2- 172.21.15.92:0/2328156029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d0101c10 0x7f32d0102010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009900 tx=0x7f32c002a410 comp rx=0 tx=0).stop 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 -- 172.21.15.92:0/2328156029 shutdown_connections 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 --2- 172.21.15.92:0/2328156029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d01037d0 0x7f32d010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 --2- 172.21.15.92:0/2328156029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:24.681 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 --2- 172.21.15.92:0/2328156029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d0101c10 0x7f32d0102010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 -- 172.21.15.92:0/2328156029 >> 172.21.15.92:0/2328156029 conn(0x7f32d00fd3c0 msgr2=0x7f32d00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.663+0000 7f32d5f23640 1 -- 172.21.15.92:0/2328156029 shutdown_connections 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 -- 172.21.15.92:0/2328156029 wait complete. 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 Processor -- start 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 -- start start 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d0101c10 0x7f32d019a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010d8f0 con 0x7f32d0102e10 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010d770 con 0x7f32d0101c10 2024-09-12T02:11:24.682 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.664+0000 7f32d5f23640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f32d010da70 con 0x7f32d01037d0 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54274/0 (socket says 172.21.15.92:54274) 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 -- 172.21.15.92:0/169166499 learned_addr learned my addr 172.21.15.92:0/169166499 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cf7fe640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d0101c10 0x7f32d019a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32ceffd640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 -- 172.21.15.92:0/169166499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d0101c10 msgr2=0x7f32d019a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d0101c10 0x7f32d019a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 -- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 msgr2=0x7f32d019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:24.683 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 -- 172.21.15.92:0/169166499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d01a1d90 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32ceffd640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32cffff640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b970 tx=0x7f32c800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8019070 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d01a2080 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c800cb40 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8007650 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.665+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d01a25f0 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.666+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d01a28b0 con 0x7f32d01037d0 2024-09-12T02:11:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.667+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f32c801c070 con 0x7f32d01037d0 2024-09-12T02:11:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.668+0000 7f32ccff9640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 0x7f32b80a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.668+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f32c80ccc30 con 0x7f32d01037d0 2024-09-12T02:11:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.668+0000 7f32cf7fe640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 0x7f32b80a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.668+0000 7f32cf7fe640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 0x7f32b80a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005ab0 tx=0x7f32c0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:24.685 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.672+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32c809aeb0 con 0x7f32d01037d0 2024-09-12T02:11:24.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:24.847+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f32d01a2a80 con 0x7f32b80a2b20 2024-09-12T02:11:25.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:11:24.597+0000 7f81f7345640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T02:11:25.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:24 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:11:24.600+0000 7f81ee951640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:11:25.500 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi026 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='osd.1 ' entity='osd.1' 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:25.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:25.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi026 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='osd.1 ' entity='osd.1' 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:25.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi026 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi026 to 17092M 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='osd.1 ' entity='osd.1' 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:26.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:11:26.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:11:26.439 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:26.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:26.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:26.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] boot 2024-09-12T02:11:26.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:11:26.440 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:26 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] boot 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:11:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:26 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:26.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:11:26.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:11:26.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:11:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: osd.1 [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] boot 2024-09-12T02:11:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:11:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:26 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:11:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/1133315064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:27 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:27 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]': finished 2024-09-12T02:11:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:27 smithi026 ceph-mon[35496]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:11:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:27.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/1133315064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:27 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:27 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]': finished 2024-09-12T02:11:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:27 smithi003 ceph-mon[31476]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:11:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:27.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/1133315064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:27 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]: dispatch 2024-09-12T02:11:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:27 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e76989d-dcc2-45a3-8f36-eb08674b7fdc"}]': finished 2024-09-12T02:11:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:27 smithi092 ceph-mon[36089]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T02:11:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:28.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:28 smithi026 ceph-mon[35496]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:28.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:28 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3838156993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:28.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:28 smithi003 ceph-mon[31476]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:28.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:28 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3838156993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:28.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:28 smithi092 ceph-mon[36089]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:28.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:28 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3838156993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:30.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:30 smithi026 ceph-mon[35496]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:30.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:30 smithi003 ceph-mon[31476]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:30.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:30 smithi092 ceph-mon[36089]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:32.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:32 smithi026 ceph-mon[35496]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:32.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:32 smithi003 ceph-mon[31476]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:32.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:32 smithi092 ceph-mon[36089]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:34.513 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:34 smithi092 ceph-mon[36089]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:34.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:34 smithi026 ceph-mon[35496]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:34.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:34 smithi003 ceph-mon[31476]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:36.320 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:36 smithi092 ceph-mon[36089]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:36.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:11:36.321 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:36.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:36 smithi026 ceph-mon[35496]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:11:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:36.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:36 smithi003 ceph-mon[31476]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:36.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:11:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:37.491 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:37 smithi092 ceph-mon[36089]: Deploying daemon osd.2 on smithi092 2024-09-12T02:11:37.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:37 smithi026 ceph-mon[35496]: Deploying daemon osd.2 on smithi092 2024-09-12T02:11:37.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:37 smithi003 ceph-mon[31476]: Deploying daemon osd.2 on smithi092 2024-09-12T02:11:38.408 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:38 smithi092 ceph-mon[36089]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:38.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:38 smithi026 ceph-mon[35496]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:38.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:38 smithi003 ceph-mon[31476]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:40.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:40 smithi026 ceph-mon[35496]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:40.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:40 smithi003 ceph-mon[31476]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:40.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:40 smithi092 ceph-mon[36089]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:42.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:42 smithi092 ceph-mon[36089]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:42.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:42.501 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:42.502 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:42.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:42 smithi026 ceph-mon[35496]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:42.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:42.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:42.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:42.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:42 smithi003 ceph-mon[31476]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:42.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:42.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:42.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:44.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:44 smithi026 ceph-mon[35496]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:44.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:44 smithi003 ceph-mon[31476]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:44.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:44 smithi092 ceph-mon[36089]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:45.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:45.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.705 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:45.706 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:45.706 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.021 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.020+0000 7f32ccff9640 1 -- 172.21.15.92:0/169166499 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32d01a2a80 con 0x7f32b80a2b20 2024-09-12T02:11:46.021 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 2 on host 'smithi092' 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 msgr2=0x7f32b80a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 0x7f32b80a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32c0005ab0 tx=0x7f32c0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 msgr2=0x7f32d01a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f32c800b970 tx=0x7f32c800be30 comp rx=0 tx=0).stop 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 shutdown_connections 2024-09-12T02:11:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32b80a2b20 0x7f32b80a4fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32d01037d0 0x7f32d01a1720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32d0102e10 0x7f32d019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 --2- 172.21.15.92:0/169166499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d0101c10 0x7f32d019a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 >> 172.21.15.92:0/169166499 conn(0x7f32d00fd3c0 msgr2=0x7f32d0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 shutdown_connections 2024-09-12T02:11:46.024 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:11:46.023+0000 7f32d5f23640 1 -- 172.21.15.92:0/169166499 wait complete. 2024-09-12T02:11:46.600 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:46 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:46.600 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:46 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.600 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:46 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.627 DEBUG:teuthology.orchestra.run.smithi092:osd.2> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T02:11:46.629 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2024-09-12T02:11:46.629 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:11:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:46 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:46 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:46 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:46 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:11:46.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:46 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:46 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:46.881 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:48.099 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:48 smithi092 ceph-mon[36089]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:48.099 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:48 smithi092 ceph-mon[36089]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:48.099 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:48 smithi092 ceph-mon[36089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:48.100 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:47 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:11:47.891+0000 7f368d0ed640 -1 osd.2 0 log_to_monitors true 2024-09-12T02:11:48.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:48 smithi003 ceph-mon[31476]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:48.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:48 smithi003 ceph-mon[31476]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:48.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:48 smithi003 ceph-mon[31476]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:48 smithi026 ceph-mon[35496]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:48 smithi026 ceph-mon[35496]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:48 smithi026 ceph-mon[35496]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:11:49.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-mon[36089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:11:49.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-mon[36089]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:11:49.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:49.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-mon[36089]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.255 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-mon[36089]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:49 smithi026 ceph-mon[35496]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:11:49.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:49 smithi026 ceph-mon[35496]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:11:49.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:49.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:49 smithi026 ceph-mon[35496]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:49 smithi026 ceph-mon[35496]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.323 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:49 smithi003 ceph-mon[31476]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:11:49.324 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:49 smithi003 ceph-mon[31476]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:11:49.324 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:49.324 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:49 smithi003 ceph-mon[31476]: from='osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.324 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:49 smithi003 ceph-mon[31476]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:11:49.326 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:11:49.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2024-09-12T02:11:49.538 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:11:49.421+0000 7f3689884640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T02:11:49.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:49 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:11:49.426+0000 7f3680e90640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:11:49.610 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:11:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 -- 172.21.15.3:0/2996287066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 msgr2=0x7f0e34074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/2996287066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e34074fa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009900 tx=0x7f0e2402a410 comp rx=0 tx=0).stop 2024-09-12T02:11:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 -- 172.21.15.3:0/2996287066 shutdown_connections 2024-09-12T02:11:50.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/2996287066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e34109a70 0x7f0e3410bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/2996287066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e34074fa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/2996287066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e340761a0 0x7f0e34074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 -- 172.21.15.3:0/2996287066 >> 172.21.15.3:0/2996287066 conn(0x7f0e340fd3b0 msgr2=0x7f0e340ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 -- 172.21.15.3:0/2996287066 shutdown_connections 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.123+0000 7f0e3bdde640 1 -- 172.21.15.3:0/2996287066 wait complete. 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 Processor -- start 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 -- start start 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e340761a0 0x7f0e3419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e34109a70 0x7f0e341a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410fa70 con 0x7f0e34074b40 2024-09-12T02:11:50.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410f8f0 con 0x7f0e340761a0 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3bdde640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3410fbf0 con 0x7f0e34109a70 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e39b7a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e39b7a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60574/0 (socket says 172.21.15.3:60574) 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e39b7a640 1 -- 172.21.15.3:0/3070416364 learned_addr learned my addr 172.21.15.3:0/3070416364 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e39379640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e340761a0 0x7f0e3419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.124+0000 7f0e3a37b640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e34109a70 0x7f0e341a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 -- 172.21.15.3:0/3070416364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e34109a70 msgr2=0x7f0e341a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e34109a70 0x7f0e341a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 -- 172.21.15.3:0/3070416364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e340761a0 msgr2=0x7f0e3419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e340761a0 0x7f0e3419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:11:50.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 -- 172.21.15.3:0/3070416364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341a1d20 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e39b7a640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d770 tx=0x7f0e3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30017070 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341a1fb0 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e30005500 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30004e00 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.125+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a2460 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.126+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34108880 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.127+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f0e30004fa0 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.128+0000 7f0e2affd640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 0x7f0e200a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.128+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7f0e300cd1d0 con 0x7f0e34074b40 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.128+0000 7f0e39379640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 0x7f0e200a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.129+0000 7f0e39379640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 0x7f0e200a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e240314e0 tx=0x7f0e24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:11:50.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.132+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e3009b230 con 0x7f0e34074b40 2024-09-12T02:11:50.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='osd.2 ' entity='osd.2' 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:50.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='osd.2 ' entity='osd.2' 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:50.319+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0e34062630 con 0x7f0e200a2b20 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='osd.2 ' entity='osd.2' 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:11:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:11:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi092 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:51.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:51.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:51.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] boot 2024-09-12T02:11:51.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:11:51.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:51.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:11:51.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:11:51.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi092 2024-09-12T02:11:51.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-12T02:11:51.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] boot 2024-09-12T02:11:51.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:11:51.291 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:51.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:11:51.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:11:51.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi092 2024-09-12T02:11:51.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi092 to 17088M 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:11:51.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: osd.2 [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] boot 2024-09-12T02:11:51.517 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T02:11:51.517 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:11:52.229 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:52 smithi003 ceph-mon[31476]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:52.229 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:52 smithi003 ceph-mon[31476]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:11:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:52 smithi092 ceph-mon[36089]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:52 smithi092 ceph-mon[36089]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:11:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:52 smithi026 ceph-mon[35496]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:11:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:52 smithi026 ceph-mon[35496]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:11:53.304 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:53.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:53.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:54.027 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:53 smithi003 sudo[50323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:54.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:53 smithi003 sudo[50323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:54.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:53 smithi003 sudo[50323]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:54.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:53 smithi003 sudo[50323]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:54.340 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]: dispatch 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.341 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]': finished 2024-09-12T02:11:54.342 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:11:54.342 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:54.342 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3596088636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:54.528 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:54 smithi003 sudo[50372]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:54.528 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:54 smithi003 sudo[50372]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:54.528 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:54 smithi003 sudo[50372]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:54.528 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:11:54 smithi003 sudo[50372]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]: dispatch 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]': finished 2024-09-12T02:11:54.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:11:54.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:54.530 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3596088636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]: dispatch 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/131974333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f79efe58-167d-445a-a4c8-47c4f46c1070"}]': finished 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:54.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3596088636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:11:54.605 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44092]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:54.605 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44092]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:54.605 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44092]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:54.605 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44092]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:55.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44095]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:55.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44095]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:55.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44095]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:55.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:11:54 smithi026 sudo[44095]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:55.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:11:55.099+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0e300978a0 con 0x7f0e34074b40 2024-09-12T02:11:55.131 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:55 smithi092 ceph-mon[36089]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:11:55.131 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:55.131 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.131 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.131 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:54 smithi092 sudo[45108]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:55.132 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:54 smithi092 sudo[45108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:55.132 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:54 smithi092 sudo[45108]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:55.132 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:54 smithi092 sudo[45108]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:55.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:55 smithi026 ceph-mon[35496]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:11:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 ceph-mon[31476]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:11:55.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:11:55.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:55.538 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:55 smithi092 sudo[45111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:55.538 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:55 smithi092 sudo[45111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:55.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:55 smithi092 sudo[45111]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:55.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:11:55 smithi092 sudo[45111]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 sudo[50753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 sudo[50753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 sudo[50753]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:55 smithi003 sudo[50753]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 sudo[50939]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 sudo[50939]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 sudo[50939]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 sudo[50939]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:56.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45114]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:56.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:56.563 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45114]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:56.564 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45114]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:56.828 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:56.829 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:56.829 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:56.829 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:56.829 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 sudo[44098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:11:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:11:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:57.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45117]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 sudo[45117]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:11:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:57.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:57.099 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 sudo[44098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:57.099 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 sudo[44098]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:57.099 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:56 smithi026 sudo[44098]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:57.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 sudo[44101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:11:57.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 sudo[44101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:11:57.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 sudo[44101]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:11:57.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 sudo[44101]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:57.996 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:57 smithi003 ceph-mon[31476]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:58.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:58.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:58.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:57 smithi026 ceph-mon[35496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:11:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:57 smithi092 ceph-mon[36089]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:11:59.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:11:59 smithi003 ceph-mon[31476]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:00.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:11:59 smithi026 ceph-mon[35496]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:00.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:11:59 smithi092 ceph-mon[36089]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:01.845 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:01 smithi003 ceph-mon[31476]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:01.845 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:12:01.845 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:02.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:01 smithi026 ceph-mon[35496]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:02.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:12:02.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:02.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:01 smithi092 ceph-mon[36089]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:02.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:12:02.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:02.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:02 smithi003 ceph-mon[31476]: Deploying daemon osd.3 on smithi003 2024-09-12T02:12:03.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:02 smithi026 ceph-mon[35496]: Deploying daemon osd.3 on smithi003 2024-09-12T02:12:03.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:02 smithi092 ceph-mon[36089]: Deploying daemon osd.3 on smithi003 2024-09-12T02:12:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:03 smithi003 ceph-mon[31476]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:12:04.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:03 smithi026 ceph-mon[35496]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:12:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:03 smithi092 ceph-mon[36089]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:12:04.982 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:04 smithi003 ceph-mon[31476]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:12:05.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:04 smithi026 ceph-mon[35496]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:12:05.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:04 smithi092 ceph-mon[36089]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:12:06.658 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:06 smithi003 ceph-mon[31476]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:12:06.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:06 smithi026 ceph-mon[35496]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:12:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:06 smithi092 ceph-mon[36089]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:12:08.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:07 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:08.239 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:07 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:08.239 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:07 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:08.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:07 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:07 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:07 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:08.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:07 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:07 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:07 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:09.199 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:08 smithi003 ceph-mon[31476]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:09.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:08 smithi026 ceph-mon[35496]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:09.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:08 smithi092 ceph-mon[36089]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:11.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.376 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:11 smithi003 ceph-mon[31476]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:11 smithi026 ceph-mon[35496]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:11.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:11.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:11.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:11.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:11 smithi092 ceph-mon[36089]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:11.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.699+0000 7f0e2affd640 1 -- 172.21.15.3:0/3070416364 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e34062630 con 0x7f0e200a2b20 2024-09-12T02:12:11.699 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2024-09-12T02:12:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 msgr2=0x7f0e200a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 0x7f0e200a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e240314e0 tx=0x7f0e24000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 msgr2=0x7f0e3419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d770 tx=0x7f0e3000dc30 comp rx=0 tx=0).stop 2024-09-12T02:12:11.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 shutdown_connections 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e200a2b20 0x7f0e200a4fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e34109a70 0x7f0e341a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e340761a0 0x7f0e3419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 --2- 172.21.15.3:0/3070416364 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34074b40 0x7f0e3419a0f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 >> 172.21.15.3:0/3070416364 conn(0x7f0e340fd3b0 msgr2=0x7f0e34105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 shutdown_connections 2024-09-12T02:12:11.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:12:11.701+0000 7f0e3bdde640 1 -- 172.21.15.3:0/3070416364 wait complete. 2024-09-12T02:12:12.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:12 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:12 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:12 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.408 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T02:12:12.410 INFO:tasks.cephadm:Deploying osd.4 on smithi026 with /dev/vg_nvme/lv_3... 2024-09-12T02:12:12.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:12:12.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:12 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:12 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:12 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:12 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:12 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:12 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:12.670 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:14.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:14 smithi003 ceph-mon[31476]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:14.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:14 smithi003 ceph-mon[31476]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:12:14.479 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:12:14 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:12:14.157+0000 7fe5004e2640 -1 osd.3 0 log_to_monitors true 2024-09-12T02:12:14.498 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:14 smithi026 ceph-mon[35496]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:14.498 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:14 smithi026 ceph-mon[35496]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:12:14.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:14 smithi092 ceph-mon[36089]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:12:14.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:14 smithi092 ceph-mon[36089]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:12:15.160 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:12:15.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2024-09-12T02:12:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:15.467 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:15.468 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.468 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:15 smithi026 ceph-mon[35496]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-mon[31476]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:12:15.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:15.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:15 smithi092 ceph-mon[36089]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:12:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- 172.21.15.26:0/2490757668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c100ad0 msgr2=0x7fcd5c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/2490757668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c100ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009980 tx=0x7fcd5002c620 comp rx=0 tx=0).stop 2024-09-12T02:12:15.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- 172.21.15.26:0/2490757668 shutdown_connections 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/2490757668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c102690 0x7fcd5c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/2490757668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/2490757668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c100ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- 172.21.15.26:0/2490757668 >> 172.21.15.26:0/2490757668 conn(0x7fcd5c0fc280 msgr2=0x7fcd5c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- 172.21.15.26:0/2490757668 shutdown_connections 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- 172.21.15.26:0/2490757668 wait complete. 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 Processor -- start 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.896+0000 7fcd62bd4640 1 -- start start 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 0x7fcd5c19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10b660 con 0x7fcd5c102690 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10b4e0 con 0x7fcd5c101cd0 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd62bd4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10b7e0 con 0x7fcd5c100ad0 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52448/0 (socket says 172.21.15.26:52448) 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 -- 172.21.15.26:0/4234995890 learned_addr learned my addr 172.21.15.26:0/4234995890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd61171640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 0x7fcd5c19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd60970640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 -- 172.21.15.26:0/4234995890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c100ad0 msgr2=0x7fcd5c195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 -- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 msgr2=0x7fcd5c19d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 0x7fcd5c19d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 -- 172.21.15.26:0/4234995890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c19d940 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd61171640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 0x7fcd5c19d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.897+0000 7fcd5bfff640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d950 tx=0x7fcd4800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.898+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48017070 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.898+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19dc30 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.898+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd48009c30 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.898+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800f6b0 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.898+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c19e190 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.899+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c107f40 con 0x7fcd5c101cd0 2024-09-12T02:12:15.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.900+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcd480109d0 con 0x7fcd5c101cd0 2024-09-12T02:12:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.900+0000 7fcd59ffb640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 0x7fcd4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.901+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7fcd480cd8c0 con 0x7fcd5c101cd0 2024-09-12T02:12:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.901+0000 7fcd60970640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 0x7fcd4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.901+0000 7fcd60970640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 0x7fcd4c0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005ab0 tx=0x7fcd50000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:15.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:15.905+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd48009de0 con 0x7fcd5c101cd0 2024-09-12T02:12:16.027 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:12:15.644+0000 7fe4fc465640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T02:12:16.028 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:12:15 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:12:15.649+0000 7fe4f4285640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:12:16.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:16.092+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcd5c062630 con 0x7fcd4c0a2b80 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi003 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' 2024-09-12T02:12:16.464 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:16.465 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:16.465 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:16.465 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] boot 2024-09-12T02:12:16.465 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:12:16.465 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi003 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] boot 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:12:16.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:12:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi003 2024-09-12T02:12:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi003 to 8546M 2024-09-12T02:12:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:12:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203]' entity='osd.3' 2024-09-12T02:12:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:16.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: osd.3 [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] boot 2024-09-12T02:12:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T02:12:16.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:12:17.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:17 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:12:17.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:17 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:12:17.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:17 smithi026 ceph-mon[35496]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:17.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:17 smithi026 ceph-mon[35496]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:12:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:17 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:12:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:17 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:12:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:17 smithi003 ceph-mon[31476]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:17.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:17 smithi003 ceph-mon[31476]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:12:17.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:17 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:12:17.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:17 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:12:17.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:17 smithi092 ceph-mon[36089]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:17.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:17 smithi092 ceph-mon[36089]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:12:18.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:18 smithi026 ceph-mon[35496]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:18.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:18 smithi026 ceph-mon[35496]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:12:18.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:18 smithi003 ceph-mon[31476]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:18 smithi003 ceph-mon[31476]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:12:18.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:18 smithi092 ceph-mon[36089]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:18.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:18 smithi092 ceph-mon[36089]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T02:12:19.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/858385311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]': finished 2024-09-12T02:12:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:12:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:19 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/3536893671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:19.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/858385311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]': finished 2024-09-12T02:12:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:12:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:19 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/3536893671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:19.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/858385311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]: dispatch 2024-09-12T02:12:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e2dea9-afc3-4c05-9c71-75758b7b311b"}]': finished 2024-09-12T02:12:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:12:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:19 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/3536893671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:20.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:20 smithi026 ceph-mon[35496]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:20 smithi026 ceph-mon[35496]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:12:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:20.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:20 smithi003 ceph-mon[31476]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:20.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:20 smithi003 ceph-mon[31476]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:12:20.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:20.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:20 smithi092 ceph-mon[36089]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:20 smithi092 ceph-mon[36089]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:12:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:21.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:21 smithi026 ceph-mon[35496]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:21.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:21 smithi026 ceph-mon[35496]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:12:21.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:21.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:21 smithi003 ceph-mon[31476]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:21.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:21 smithi003 ceph-mon[31476]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:12:21.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:21.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:21 smithi092 ceph-mon[36089]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:21.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:21 smithi092 ceph-mon[36089]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:12:21.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:23.676 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:23 smithi026 ceph-mon[35496]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T02:12:23.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:23 smithi003 ceph-mon[31476]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T02:12:23.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:23 smithi092 ceph-mon[36089]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T02:12:25.690 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:25 smithi026 ceph-mon[35496]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:12:25.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:25 smithi003 ceph-mon[31476]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:12:25.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:25 smithi092 ceph-mon[36089]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:12:27.469 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:27 smithi026 ceph-mon[35496]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:12:27.469 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:12:27.469 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:27.469 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:27 smithi026 ceph-mon[35496]: Deploying daemon osd.4 on smithi026 2024-09-12T02:12:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:27 smithi092 ceph-mon[36089]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:12:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:12:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:27 smithi092 ceph-mon[36089]: Deploying daemon osd.4 on smithi026 2024-09-12T02:12:27.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:27 smithi003 ceph-mon[31476]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:12:27.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:12:27.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:27.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:27 smithi003 ceph-mon[31476]: Deploying daemon osd.4 on smithi026 2024-09-12T02:12:29.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:29 smithi026 ceph-mon[35496]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:12:29.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:29 smithi003 ceph-mon[31476]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:12:29.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:29 smithi092 ceph-mon[36089]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:12:31.703 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:31 smithi026 ceph-mon[35496]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:12:31.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:31 smithi003 ceph-mon[31476]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:12:31.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:31 smithi092 ceph-mon[36089]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:12:33.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:33 smithi026 ceph-mon[35496]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:12:33.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:33 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:33.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:33 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:33.635 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:33 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:33.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:33 smithi003 ceph-mon[31476]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:12:33.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:33 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:33.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:33 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:33.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:33 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:33.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:33 smithi092 ceph-mon[36089]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:12:33.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:33 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:33.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:33 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:33.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:33 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.491 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:35.491 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.491 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.491 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:35.491 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:35.492 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:35.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:35.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:35.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:35.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:35.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.038 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 4 on host 'smithi026' 2024-09-12T02:12:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.037+0000 7fcd59ffb640 1 -- 172.21.15.26:0/4234995890 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcd5c062630 con 0x7fcd4c0a2b80 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.039+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 msgr2=0x7fcd4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 0x7fcd4c0a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005ab0 tx=0x7fcd50000bc0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 msgr2=0x7fcd5c196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800d950 tx=0x7fcd4800de10 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 shutdown_connections 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd4c0a2b80 0x7fcd4c0a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd5c102690 0x7fcd5c19d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd5c101cd0 0x7fcd5c196260 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 --2- 172.21.15.26:0/4234995890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd5c100ad0 0x7fcd5c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 >> 172.21.15.26:0/4234995890 conn(0x7fcd5c0fc280 msgr2=0x7fcd5c105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 shutdown_connections 2024-09-12T02:12:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:12:37.040+0000 7fcd62bd4640 1 -- 172.21.15.26:0/4234995890 wait complete. 2024-09-12T02:12:37.620 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:37 smithi026 ceph-mon[35496]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:37.620 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:37.620 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.620 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.647 DEBUG:teuthology.orchestra.run.smithi026:osd.4> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T02:12:37.649 INFO:tasks.cephadm:Deploying osd.5 on smithi092 with /dev/vg_nvme/lv_3... 2024-09-12T02:12:37.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:12:37.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:37 smithi003 ceph-mon[31476]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:37.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:37.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:37 smithi092 ceph-mon[36089]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:12:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:37.912 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:38.765 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:12:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:12:38.487+0000 7f8a07a39640 -1 osd.4 0 log_to_monitors true 2024-09-12T02:12:39.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:39 smithi026 ceph-mon[35496]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:39 smithi026 ceph-mon[35496]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:39 smithi026 ceph-mon[35496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:39.709 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:39 smithi092 ceph-mon[36089]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:39.709 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:39 smithi092 ceph-mon[36089]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:39.709 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:39 smithi092 ceph-mon[36089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:39 smithi003 ceph-mon[31476]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:39 smithi003 ceph-mon[31476]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:39.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:39 smithi003 ceph-mon[31476]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:12:40.433 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:12:40.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-09-12T02:12:40.526 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:12:40.448+0000 7f8a039bc640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T02:12:40.527 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:12:40.456+0000 7f89ff7e4640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:12:40.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:12:40.528 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:40.715 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:12:40.741 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:40.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:12:40.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:12:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 -- 172.21.15.92:0/2662706047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd0102e00 msgr2=0x7f6dd0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2662706047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd0102e00 0x7f6dd0103280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009900 tx=0x7f6dc002a410 comp rx=0 tx=0).stop 2024-09-12T02:12:41.167 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 -- 172.21.15.92:0/2662706047 shutdown_connections 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2662706047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd01037c0 0x7f6dd010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2662706047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd0102e00 0x7f6dd0103280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2662706047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0101c00 0x7f6dd0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 -- 172.21.15.92:0/2662706047 >> 172.21.15.92:0/2662706047 conn(0x7f6dd00fd390 msgr2=0x7f6dd00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 -- 172.21.15.92:0/2662706047 shutdown_connections 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.160+0000 7f6dd5761640 1 -- 172.21.15.92:0/2662706047 wait complete. 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 Processor -- start 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 -- start start 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd0101c00 0x7f6dd0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 0x7f6dd0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd010d880 con 0x7f6dd0102e00 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd010d700 con 0x7f6dd0101c00 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dd5761640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd010da00 con 0x7f6dd01037c0 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dcf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.161+0000 7f6dcf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45386/0 (socket says 172.21.15.92:45386) 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 -- 172.21.15.92:0/2668119039 learned_addr learned my addr 172.21.15.92:0/2668119039 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dceffd640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd0101c00 0x7f6dd0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dce7fc640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 0x7f6dd0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 -- 172.21.15.92:0/2668119039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd0101c00 msgr2=0x7f6dd0197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd0101c00 0x7f6dd0197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 -- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 msgr2=0x7f6dd0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 0x7f6dd0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 -- 172.21.15.92:0/2668119039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd019fb00 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dce7fc640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 0x7f6dd0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dcf7fe640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800eaa0 tx=0x7f6dc800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800cdb0 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd019fdf0 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc8005020 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc80116d0 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.162+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01a02a0 con 0x7f6dd01037c0 2024-09-12T02:12:41.170 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.163+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd0108fb0 con 0x7f6dd01037c0 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.164+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6dc80054d0 con 0x7f6dd01037c0 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.165+0000 7f6db7fff640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 0x7f6db80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.165+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f6dc80ce040 con 0x7f6dd01037c0 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.165+0000 7f6dceffd640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 0x7f6db80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.166+0000 7f6dceffd640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 0x7f6db80a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:12:41.171 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.169+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6dc8016030 con 0x7f6dd01037c0 2024-09-12T02:12:41.351 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:12:41.351+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6dd0062630 con 0x7f6db80a2bb0 2024-09-12T02:12:41.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:41.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi026 2024-09-12T02:12:41.736 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:41.737 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.738 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi026 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi026 2024-09-12T02:12:41.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi026 to 8546M 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:41.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:42.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:42 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:12:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:42 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:12:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:42 smithi026 ceph-mon[35496]: osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] boot 2024-09-12T02:12:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:42 smithi026 ceph-mon[35496]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:12:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:42 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:12:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:42 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:12:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:42 smithi003 ceph-mon[31476]: osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] boot 2024-09-12T02:12:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:42 smithi003 ceph-mon[31476]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:12:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:42 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:12:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:42 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:12:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:42 smithi092 ceph-mon[36089]: osd.4 [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] boot 2024-09-12T02:12:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:42 smithi092 ceph-mon[36089]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:12:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:12:43.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:43 smithi026 ceph-mon[35496]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:43.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:43 smithi026 ceph-mon[35496]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:12:43.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:43 smithi003 ceph-mon[31476]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:43.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:43 smithi003 ceph-mon[31476]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:12:43.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:43 smithi092 ceph-mon[36089]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:12:43.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:43 smithi092 ceph-mon[36089]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T02:12:44.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3529617571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]': finished 2024-09-12T02:12:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:12:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:12:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:44 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3295136029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:44.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3529617571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]': finished 2024-09-12T02:12:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:12:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:12:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:44 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3295136029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3529617571' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]: dispatch 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a9b28bd-0597-4710-b614-d90867985f7f"}]': finished 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:12:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:44 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3295136029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:12:46.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:45 smithi026 ceph-mon[35496]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:46.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:45 smithi003 ceph-mon[31476]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:46.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:45 smithi092 ceph-mon[36089]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:47 smithi026 ceph-mon[35496]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:48.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:47 smithi003 ceph-mon[31476]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:48.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:47 smithi092 ceph-mon[36089]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:49 smithi092 ceph-mon[36089]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:49 smithi026 ceph-mon[35496]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:49 smithi003 ceph-mon[31476]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:51 smithi092 ceph-mon[36089]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:52.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:51 smithi026 ceph-mon[35496]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:51 smithi003 ceph-mon[31476]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:53.114 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:12:53.115 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:53.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:12:53.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:53.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:12:53.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:12:54.161 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:53 smithi092 ceph-mon[36089]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:54.161 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:53 smithi092 ceph-mon[36089]: Deploying daemon osd.5 on smithi092 2024-09-12T02:12:54.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:53 smithi026 ceph-mon[35496]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:54.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:53 smithi026 ceph-mon[35496]: Deploying daemon osd.5 on smithi092 2024-09-12T02:12:54.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:53 smithi003 ceph-mon[31476]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:54.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:53 smithi003 ceph-mon[31476]: Deploying daemon osd.5 on smithi092 2024-09-12T02:12:56.104 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:55 smithi092 ceph-mon[36089]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:56.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:55 smithi026 ceph-mon[35496]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:56.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:55 smithi003 ceph-mon[31476]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:58.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:57 smithi026 ceph-mon[35496]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:58.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:57 smithi092 ceph-mon[36089]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:12:58.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:57 smithi003 ceph-mon[31476]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:00.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:59 smithi092 ceph-mon[36089]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:00.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:00.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:00.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:12:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:59 smithi026 ceph-mon[35496]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:12:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:00.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:59 smithi003 ceph-mon[31476]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:00.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:12:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:02.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:02.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:02.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:02.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:03.081 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.079+0000 7f6db7fff640 1 -- 172.21.15.92:0/2668119039 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6dd0062630 con 0x7f6db80a2bb0 2024-09-12T02:13:03.081 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 5 on host 'smithi092' 2024-09-12T02:13:03.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 msgr2=0x7f6db80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:03.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 0x7f6db80a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005fd0 tx=0x7f6dc400b410 comp rx=0 tx=0).stop 2024-09-12T02:13:03.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 msgr2=0x7f6dd019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800eaa0 tx=0x7f6dc800ef60 comp rx=0 tx=0).stop 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 shutdown_connections 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6db80a2bb0 0x7f6db80a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6dd01037c0 0x7f6dd019f490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6dd0102e00 0x7f6dd0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 --2- 172.21.15.92:0/2668119039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dd0101c00 0x7f6dd0197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 >> 172.21.15.92:0/2668119039 conn(0x7f6dd00fd390 msgr2=0x7f6dd0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 shutdown_connections 2024-09-12T02:13:03.084 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:03.082+0000 7f6dd5761640 1 -- 172.21.15.92:0/2668119039 wait complete. 2024-09-12T02:13:03.765 DEBUG:teuthology.orchestra.run.smithi092:osd.5> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T02:13:03.768 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2024-09-12T02:13:03.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:13:04.023 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:03 smithi092 ceph-mon[36089]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.144 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:03 smithi003 ceph-mon[31476]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:04.144 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.144 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.144 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:03 smithi026 ceph-mon[35496]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:04.969 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:04 smithi092 ceph-mon[36089]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:04.969 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:04 smithi092 ceph-mon[36089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:04.969 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:13:04 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:13:04.675+0000 7f686f8ed640 -1 osd.5 0 log_to_monitors true 2024-09-12T02:13:05.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:04 smithi003 ceph-mon[31476]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:05.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:04 smithi003 ceph-mon[31476]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:05.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:04 smithi026 ceph-mon[35496]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:05.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:04 smithi026 ceph-mon[35496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:13:06.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:06.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:13:06.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.247 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:13:06.248 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:06.248 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:05 smithi092 ceph-mon[36089]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:06.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:05 smithi003 ceph-mon[31476]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:13:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: from='osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:13:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:05 smithi026 ceph-mon[35496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:13:06.539 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:13:06.245+0000 7f686b870640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T02:13:06.539 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:13:06.254+0000 7f6863690640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:13:06.558 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:13:06.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2024-09-12T02:13:06.854 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:13:07.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='osd.5 ' entity='osd.5' 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.155 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] boot 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:13:07.156 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='osd.5 ' entity='osd.5' 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:07.157 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:07.158 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.158 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] boot 2024-09-12T02:13:07.158 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:13:07.158 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:13:07.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='osd.5 ' entity='osd.5' 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:07.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: osd.5 [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] boot 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:13:07.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:13:07.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 -- 172.21.15.3:0/968086510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6658101c00 msgr2=0x7f6658102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 --2- 172.21.15.3:0/968086510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6658101c00 0x7f6658102000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6648009900 tx=0x7f664802a410 comp rx=0 tx=0).stop 2024-09-12T02:13:07.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 -- 172.21.15.3:0/968086510 shutdown_connections 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 --2- 172.21.15.3:0/968086510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66581037c0 0x7f665810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 --2- 172.21.15.3:0/968086510 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658102e00 0x7f6658103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.340+0000 7f665e1e5640 1 --2- 172.21.15.3:0/968086510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6658101c00 0x7f6658102000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.341+0000 7f665e1e5640 1 -- 172.21.15.3:0/968086510 >> 172.21.15.3:0/968086510 conn(0x7f66580fd390 msgr2=0x7f66580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.341+0000 7f665e1e5640 1 -- 172.21.15.3:0/968086510 shutdown_connections 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.341+0000 7f665e1e5640 1 -- 172.21.15.3:0/968086510 wait complete. 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.341+0000 7f665e1e5640 1 Processor -- start 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.341+0000 7f665e1e5640 1 -- start start 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658101c00 0x7f6658075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 0x7f66580745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f665810d880 con 0x7f66581037c0 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f665810d700 con 0x7f6658102e00 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f665e1e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f665810da00 con 0x7f6658101c00 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37378/0 (socket says 172.21.15.3:37378) 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 -- 172.21.15.3:0/25722563 learned_addr learned my addr 172.21.15.3:0/25722563 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f66577fe640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658101c00 0x7f6658075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6656ffd640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 0x7f66580745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 -- 172.21.15.3:0/25722563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658101c00 msgr2=0x7f6658075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658101c00 0x7f6658075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 -- 172.21.15.3:0/25722563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 msgr2=0x7f66580745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:07.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 0x7f66580745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 -- 172.21.15.3:0/25722563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66580753e0 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6656ffd640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 0x7f66580745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.342+0000 7f6657fff640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f665000eaa0 tx=0x7f665000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.343+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665000cdb0 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.343+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6650005020 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.343+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66581a2060 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.343+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500116b0 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.343+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66581a2520 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.344+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658108ff0 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.345+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6650005560 con 0x7f66581037c0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.345+0000 7f6654ff9640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 0x7f66400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.346+0000 7f66577fe640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 0x7f66400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:07.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.346+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f66500ce8a0 con 0x7f66581037c0 2024-09-12T02:13:07.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.346+0000 7f66577fe640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 0x7f66400a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6648006010 tx=0x7f6648000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:07.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.349+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6650016030 con 0x7f66581037c0 2024-09-12T02:13:07.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:07.534+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6658062630 con 0x7f66400a2b80 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi092 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:08.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:08 smithi026 ceph-mon[35496]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:13:08.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi092 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:08 smithi003 ceph-mon[31476]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi092 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi092 to 8544M 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:08 smithi092 ceph-mon[36089]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T02:13:09.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:09 smithi026 ceph-mon[35496]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:09.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:09 smithi026 ceph-mon[35496]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:13:09.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:09 smithi003 ceph-mon[31476]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:09.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:09 smithi003 ceph-mon[31476]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:13:09.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:09 smithi092 ceph-mon[36089]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:09.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:09 smithi092 ceph-mon[36089]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T02:13:10.491 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:10 smithi003 ceph-mon[31476]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:10.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:10 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]: dispatch 2024-09-12T02:13:10.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:10 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]': finished 2024-09-12T02:13:10.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:10 smithi003 ceph-mon[31476]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:13:10.492 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:10 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:10 smithi026 ceph-mon[35496]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:10 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]: dispatch 2024-09-12T02:13:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:10 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]': finished 2024-09-12T02:13:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:10 smithi026 ceph-mon[35496]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:13:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:10 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:10 smithi092 ceph-mon[36089]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:10 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]: dispatch 2024-09-12T02:13:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:10 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3576943646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c4272b-d682-4423-924c-8500633b6e24"}]': finished 2024-09-12T02:13:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:10 smithi092 ceph-mon[36089]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:13:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:10 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:11.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:11 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3710652690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:11 smithi026 ceph-mon[35496]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:13:11.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:11.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:11 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3710652690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:11.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:11 smithi003 ceph-mon[31476]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:13:11.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:11.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:11 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3710652690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:11.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:11 smithi092 ceph-mon[36089]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:13:11.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:12.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:12 smithi026 ceph-mon[35496]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:12.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:12 smithi003 ceph-mon[31476]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:12.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:12 smithi092 ceph-mon[36089]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:14.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:14 smithi003 ceph-mon[31476]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T02:13:14.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:14 smithi092 ceph-mon[36089]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T02:13:14.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:14 smithi026 ceph-mon[35496]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-12T02:13:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:16 smithi003 ceph-mon[31476]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:13:16.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:16 smithi092 ceph-mon[36089]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:13:16.567 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:16 smithi026 ceph-mon[35496]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:13:18.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:18 smithi003 ceph-mon[31476]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:13:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:18 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:13:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:18 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:18.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:18 smithi092 ceph-mon[36089]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:13:18.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:18 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:13:18.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:18 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:18 smithi026 ceph-mon[35496]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:13:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:18 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:13:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:18 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:19.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:19 smithi003 ceph-mon[31476]: Deploying daemon osd.6 on smithi003 2024-09-12T02:13:19.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:19 smithi092 ceph-mon[36089]: Deploying daemon osd.6 on smithi003 2024-09-12T02:13:19.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:19 smithi026 ceph-mon[35496]: Deploying daemon osd.6 on smithi003 2024-09-12T02:13:20.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:20 smithi003 ceph-mon[31476]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:13:20.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:20 smithi092 ceph-mon[36089]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:13:20.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:20 smithi026 ceph-mon[35496]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T02:13:22.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:22 smithi092 ceph-mon[36089]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:13:22.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:22 smithi003 ceph-mon[31476]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:13:22.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:22 smithi026 ceph-mon[35496]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:13:24.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:24 smithi003 ceph-mon[31476]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:13:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:24.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:24 smithi092 ceph-mon[36089]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:13:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:24.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:24 smithi026 ceph-mon[35496]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T02:13:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:26.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:26 smithi003 ceph-mon[31476]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:26 smithi092 ceph-mon[36089]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:26.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:26 smithi026 ceph-mon[35496]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:27.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.860+0000 7f6654ff9640 1 -- 172.21.15.3:0/25722563 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6658062630 con 0x7f66400a2b80 2024-09-12T02:13:27.861 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 msgr2=0x7f66400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 0x7f66400a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6648006010 tx=0x7f6648000bc0 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 msgr2=0x7f6658074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f665000eaa0 tx=0x7f665000ef60 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 shutdown_connections 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f66400a2b80 0x7f66400a5040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f66581037c0 0x7f6658074b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6658102e00 0x7f66580745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 --2- 172.21.15.3:0/25722563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6658101c00 0x7f6658075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:27.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 >> 172.21.15.3:0/25722563 conn(0x7f66580fd390 msgr2=0x7f66581061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:27.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 shutdown_connections 2024-09-12T02:13:27.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:13:27.863+0000 7f665e1e5640 1 -- 172.21.15.3:0/25722563 wait complete. 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:28.171 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.487 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T02:13:28.489 INFO:tasks.cephadm:Deploying osd.7 on smithi026 with /dev/vg_nvme/lv_2... 2024-09-12T02:13:28.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:13:28.751 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:13:28.959 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:28 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:28.959 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:28 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:28.959 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:28 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:29.115 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:28 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:29.115 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:28 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:29.115 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:28 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:29.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:28 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:29.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:28 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:29.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:28 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:30.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:29 smithi003 ceph-mon[31476]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:30.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:29 smithi003 ceph-mon[31476]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:13:30.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:13:29 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:13:29.837+0000 7f8be3fac640 -1 osd.6 0 log_to_monitors true 2024-09-12T02:13:30.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:29 smithi026 ceph-mon[35496]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:30.145 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:29 smithi026 ceph-mon[35496]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:13:30.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:29 smithi092 ceph-mon[36089]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:30.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:29 smithi092 ceph-mon[36089]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:13:31.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:13:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:30 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.027 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:13:30.867+0000 7f8bdff2f640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T02:13:31.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:13:30.880+0000 7f8bdbd57640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:13:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:30 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: from='osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T02:13:31.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:30 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:31.376 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:13:31.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2024-09-12T02:13:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:13:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 -- 172.21.15.26:0/3737381761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4102690 msgr2=0x7fdbb4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 --2- 172.21.15.26:0/3737381761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4102690 0x7fdbb4108f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ca0 tx=0x7fdbac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 -- 172.21.15.26:0/3737381761 shutdown_connections 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 --2- 172.21.15.26:0/3737381761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4102690 0x7fdbb4108f20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 --2- 172.21.15.26:0/3737381761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 0x7fdbb4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 --2- 172.21.15.26:0/3737381761 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 -- 172.21.15.26:0/3737381761 >> 172.21.15.26:0/3737381761 conn(0x7fdbb40fc280 msgr2=0x7fdbb40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 -- 172.21.15.26:0/3737381761 shutdown_connections 2024-09-12T02:13:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.094+0000 7fdbbba46640 1 -- 172.21.15.26:0/3737381761 wait complete. 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 Processor -- start 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 -- start start 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 0x7fdbb41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 0x7fdbb41035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410c750 con 0x7fdbb4102690 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410c5d0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbbba46640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb410c8d0 con 0x7fdbb4101cd0 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44650/0 (socket says 172.21.15.26:44650) 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 -- 172.21.15.26:0/2427069798 learned_addr learned my addr 172.21.15.26:0/2427069798 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-12T02:13:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb8fe1640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 0x7fdbb41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb9fe3640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 0x7fdbb41035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 -- 172.21.15.26:0/2427069798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 msgr2=0x7fdbb41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 0x7fdbb41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 -- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 msgr2=0x7fdbb41035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 0x7fdbb41035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 -- 172.21.15.26:0/2427069798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb4103eb0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb9fe3640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 0x7fdbb41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.095+0000 7fdbb97e2640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000e9c0 tx=0x7fdbb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.096+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000cd80 con 0x7fdbb4100ad0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.096+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb41a1ff0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.096+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb00035f0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.096+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0010690 con 0x7fdbb4100ad0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.096+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41a2450 con 0x7fdbb4100ad0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.097+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb41a2740 con 0x7fdbb4100ad0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.098+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdbb00037a0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.098+0000 7fdbaa7fc640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.099+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fdbb00ce780 con 0x7fdbb4100ad0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.099+0000 7fdbb8fe1640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.099+0000 7fdbb8fe1640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41047e0 tx=0x7fdba4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.103+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbb009c3f0 con 0x7fdbb4100ad0 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:32.291+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdbb41a2910 con 0x7fdba00a2b80 2024-09-12T02:13:32.978 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:13:32.978 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi003 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] boot 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:32.979 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:32 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:33.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:13:33.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:13:33.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi003 2024-09-12T02:13:33.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-12T02:13:33.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] boot 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:33.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:32 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:33.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi003 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi003 to 5697M 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: osd.6 [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] boot 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:33.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:32 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:33.996 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:33 smithi026 ceph-mon[35496]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:33.996 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:33 smithi026 ceph-mon[35496]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:13:34.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:33 smithi003 ceph-mon[31476]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:34.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:33 smithi003 ceph-mon[31476]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:13:34.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:33 smithi092 ceph-mon[36089]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:13:34.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:33 smithi092 ceph-mon[36089]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T02:13:35.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:34 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/649178865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:34 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:34 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]': finished 2024-09-12T02:13:35.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:34 smithi026 ceph-mon[35496]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:13:35.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:34 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:35.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:34 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/649178865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:34 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:34 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]': finished 2024-09-12T02:13:35.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:34 smithi003 ceph-mon[31476]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:13:35.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:34 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:35.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:34 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/649178865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:34 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]: dispatch 2024-09-12T02:13:35.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:34 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0"}]': finished 2024-09-12T02:13:35.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:34 smithi092 ceph-mon[36089]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:13:35.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:34 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:36.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:35 smithi026 ceph-mon[35496]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:36.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:35 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/2052226934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:36.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:35 smithi003 ceph-mon[31476]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:36.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:35 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/2052226934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:36.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:35 smithi092 ceph-mon[36089]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:36.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:35 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/2052226934' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:13:38.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:37 smithi026 ceph-mon[35496]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:38.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:37 smithi003 ceph-mon[31476]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:38.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:37 smithi092 ceph-mon[36089]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:40.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:39 smithi026 ceph-mon[35496]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:39 smithi003 ceph-mon[31476]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:40.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:39 smithi092 ceph-mon[36089]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:41.966 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:41 smithi026 ceph-mon[35496]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:41 smithi003 ceph-mon[31476]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:42.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:41 smithi092 ceph-mon[36089]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:44.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:43 smithi026 ceph-mon[35496]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:13:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:43 smithi026 ceph-mon[35496]: Deploying daemon osd.7 on smithi026 2024-09-12T02:13:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:43 smithi003 ceph-mon[31476]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:13:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:43 smithi003 ceph-mon[31476]: Deploying daemon osd.7 on smithi026 2024-09-12T02:13:44.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:43 smithi092 ceph-mon[36089]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:44.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:13:44.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:44.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:43 smithi092 ceph-mon[36089]: Deploying daemon osd.7 on smithi026 2024-09-12T02:13:46.113 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:45 smithi026 ceph-mon[35496]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:46.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:45 smithi003 ceph-mon[31476]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:46.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:45 smithi092 ceph-mon[36089]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:48.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:47 smithi026 ceph-mon[35496]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:48.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:47 smithi003 ceph-mon[31476]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:48.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:47 smithi092 ceph-mon[36089]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:50.245 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:49 smithi026 ceph-mon[35496]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:50.245 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:50.245 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:50.245 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:49 smithi003 ceph-mon[31476]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:50.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:49 smithi092 ceph-mon[36089]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:50.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:50.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:52.182 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:51 smithi026 ceph-mon[35496]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:51 smithi003 ceph-mon[31476]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:52.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:51 smithi092 ceph-mon[36089]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:53.361 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 7 on host 'smithi026' 2024-09-12T02:13:53.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.360+0000 7fdbaa7fc640 1 -- 172.21.15.26:0/2427069798 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbb41a2910 con 0x7fdba00a2b80 2024-09-12T02:13:53.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.362+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 msgr2=0x7fdba00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.362+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdbb41047e0 tx=0x7fdba4005f90 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.362+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 msgr2=0x7fdbb4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.362+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000e9c0 tx=0x7fdbb000ee80 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 shutdown_connections 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4102690 0x7fdbb41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4101cd0 0x7fdbb41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 --2- 172.21.15.26:0/2427069798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4100ad0 0x7fdbb4104a60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 >> 172.21.15.26:0/2427069798 conn(0x7fdbb40fc280 msgr2=0x7fdbb4069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 shutdown_connections 2024-09-12T02:13:53.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:13:53.363+0000 7fdbbba46640 1 -- 172.21.15.26:0/2427069798 wait complete. 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.732 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:53 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:53 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:53 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:54.067 DEBUG:teuthology.orchestra.run.smithi026:osd.7> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T02:13:54.070 INFO:tasks.cephadm:Deploying osd.8 on smithi092 with /dev/vg_nvme/lv_2... 2024-09-12T02:13:54.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:13:54.327 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:13:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:55 smithi026 ceph-mon[35496]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:55 smithi026 ceph-mon[35496]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:55.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:55 smithi026 ceph-mon[35496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:55.515 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:13:55 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:13:55.235+0000 7febce440640 -1 osd.7 0 log_to_monitors true 2024-09-12T02:13:55.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:55 smithi092 ceph-mon[36089]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:55.674 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:55 smithi092 ceph-mon[36089]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:55.675 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:55 smithi092 ceph-mon[36089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:55 smithi003 ceph-mon[31476]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:55 smithi003 ceph-mon[31476]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:55 smithi003 ceph-mon[31476]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:13:56.628 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-mon[35496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:13:56.628 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-mon[35496]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:56.628 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-mon[35496]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:13:56.628 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:56.628 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-mon[35496]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:56.629 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:13:56.440+0000 7febca3c3640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T02:13:56.629 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:13:56 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:13:56.448+0000 7febc61eb640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:13:56.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:56 smithi092 ceph-mon[36089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:13:56.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:56 smithi092 ceph-mon[36089]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:56.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:56 smithi092 ceph-mon[36089]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:13:56.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:56.742 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:56 smithi092 ceph-mon[36089]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:56.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:56 smithi003 ceph-mon[31476]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:13:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:56 smithi003 ceph-mon[31476]: from='osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:56 smithi003 ceph-mon[31476]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:13:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:56 smithi003 ceph-mon[31476]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:13:57.176 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:13:57.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-09-12T02:13:57.445 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:13:57.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:57.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:13:57.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi026 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:57.699 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi026 2024-09-12T02:13:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi026 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi026 to 5697M 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:13:57.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:13:57.926 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.901+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/355892316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48101c00 msgr2=0x7f2a48102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.901+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/355892316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48101c00 0x7f2a48102000 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009900 tx=0x7f2a3802a410 comp rx=0 tx=0).stop 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/355892316 shutdown_connections 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/355892316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a481037c0 0x7f2a4810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/355892316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a48102e00 0x7f2a48103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/355892316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48101c00 0x7f2a48102000 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/355892316 >> 172.21.15.92:0/355892316 conn(0x7f2a480fd390 msgr2=0x7f2a480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/355892316 shutdown_connections 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/355892316 wait complete. 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 Processor -- start 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- start start 2024-09-12T02:13:57.927 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48101c00 0x7f2a48075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a481037c0 0x7f2a48074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810d880 con 0x7f2a48101c00 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810d700 con 0x7f2a481037c0 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a4d7f3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4810da00 con 0x7f2a48102e00 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.902+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:49260/0 (socket says 172.21.15.92:49260) 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 -- 172.21.15.92:0/1115493349 learned_addr learned my addr 172.21.15.92:0/1115493349 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a46ffd640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48101c00 0x7f2a48075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a477fe640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a481037c0 0x7f2a48074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:57.928 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 -- 172.21.15.92:0/1115493349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a481037c0 msgr2=0x7f2a48074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a481037c0 0x7f2a48074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 -- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48101c00 msgr2=0x7f2a48075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48101c00 0x7f2a48075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 -- 172.21.15.92:0/1115493349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480753e0 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a3ffff640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000e9f0 tx=0x7f2a3000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000cd80 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a30005500 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a481a20b0 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30010690 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.903+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a481a2570 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.904+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a48109000 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.905+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2a300046c0 con 0x7f2a48102e00 2024-09-12T02:13:57.929 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.906+0000 7f2a44ff9640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 0x7f2a340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:13:57.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.906+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7f2a300ceab0 con 0x7f2a48102e00 2024-09-12T02:13:57.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.906+0000 7f2a46ffd640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 0x7f2a340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:13:57.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.907+0000 7f2a46ffd640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 0x7f2a340a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005ab0 tx=0x7f2a38000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:13:57.930 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:57.909+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a300541b0 con 0x7f2a48102e00 2024-09-12T02:13:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:13:58.085+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2a48062630 con 0x7f2a340a2b80 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] boot 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:58.754 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:58.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:58.755 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:58 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:58.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] boot 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:58 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: osd.7 [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] boot 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:13:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:58 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:13:59.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:59 smithi092 ceph-mon[36089]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:59.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:13:59 smithi092 ceph-mon[36089]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:13:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:59 smithi026 ceph-mon[35496]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:13:59 smithi026 ceph-mon[35496]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:13:59.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:59 smithi003 ceph-mon[31476]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:13:59.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:13:59 smithi003 ceph-mon[31476]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T02:14:00.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:00 smithi092 ceph-mon[36089]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:14:00.461 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:00 smithi092 ceph-mon[36089]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:14:00.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:00 smithi026 ceph-mon[35496]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:14:00.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:00 smithi026 ceph-mon[35496]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:14:00.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:00 smithi003 ceph-mon[31476]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T02:14:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:00 smithi003 ceph-mon[31476]: osdmap e52: 8 total, 8 up, 8 in 2024-09-12T02:14:01.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/777885553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]': finished 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:01 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/1573769920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/777885553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]': finished 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:01.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/1573769920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/777885553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]: dispatch 2024-09-12T02:14:01.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2c1241a-ec0f-4db8-b5d4-0e8895ddc469"}]': finished 2024-09-12T02:14:01.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:14:01.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:01.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:01 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/1573769920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:03 smithi026 ceph-mon[35496]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:03.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:03 smithi003 ceph-mon[31476]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:03.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:03 smithi092 ceph-mon[36089]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:05.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:05 smithi026 ceph-mon[35496]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:05.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:05 smithi003 ceph-mon[31476]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:05.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:05 smithi092 ceph-mon[36089]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:07.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:07 smithi026 ceph-mon[35496]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:07.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:07 smithi003 ceph-mon[31476]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:07.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:07 smithi092 ceph-mon[36089]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:09.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:09 smithi092 ceph-mon[36089]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:09.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:14:09.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:09.727 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:09 smithi092 ceph-mon[36089]: Deploying daemon osd.8 on smithi092 2024-09-12T02:14:09.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:09 smithi026 ceph-mon[35496]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:09.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:14:09.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:09.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:09 smithi026 ceph-mon[35496]: Deploying daemon osd.8 on smithi092 2024-09-12T02:14:09.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:09 smithi003 ceph-mon[31476]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:09.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:14:09.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:09.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:09 smithi003 ceph-mon[31476]: Deploying daemon osd.8 on smithi092 2024-09-12T02:14:11.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:11 smithi026 ceph-mon[35496]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:11.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:11 smithi003 ceph-mon[31476]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:11.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:11 smithi092 ceph-mon[36089]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:13.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:13 smithi026 ceph-mon[35496]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:13.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:13 smithi003 ceph-mon[31476]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:13.784 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:13 smithi092 ceph-mon[36089]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:15 smithi026 ceph-mon[35496]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:15 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:15.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:15 smithi003 ceph-mon[31476]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:15.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:15.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:15.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:15 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:15 smithi092 ceph-mon[36089]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:15 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:17.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:17 smithi026 ceph-mon[35496]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:17.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:17 smithi003 ceph-mon[31476]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:17.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:17 smithi092 ceph-mon[36089]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:19.146 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.146+0000 7f2a44ff9640 1 -- 172.21.15.92:0/1115493349 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a48062630 con 0x7f2a340a2b80 2024-09-12T02:14:19.147 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 8 on host 'smithi092' 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 msgr2=0x7f2a340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 0x7f2a340a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005ab0 tx=0x7f2a38000bc0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 msgr2=0x7f2a480745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000e9f0 tx=0x7f2a3000edf0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 shutdown_connections 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a340a2b80 0x7f2a340a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a481037c0 0x7f2a48074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a48102e00 0x7f2a480745e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.149 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 --2- 172.21.15.92:0/1115493349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a48101c00 0x7f2a48075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 >> 172.21.15.92:0/1115493349 conn(0x7f2a480fd390 msgr2=0x7f2a481061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 shutdown_connections 2024-09-12T02:14:19.150 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:14:19.148+0000 7f2a4d7f3640 1 -- 172.21.15.92:0/1115493349 wait complete. 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:19.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:19 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:19 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:19 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:19.943 DEBUG:teuthology.orchestra.run.smithi092:osd.8> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T02:14:19.946 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2024-09-12T02:14:19.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:14:20.212 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:20.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:20 smithi003 ceph-mon[31476]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:20.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:20 smithi026 ceph-mon[35496]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:20.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:20 smithi092 ceph-mon[36089]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:21 smithi092 ceph-mon[36089]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:21.279 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:21 smithi092 ceph-mon[36089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:21.280 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:14:21 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:14:21.195+0000 7f98afb3d640 -1 osd.8 0 log_to_monitors true 2024-09-12T02:14:21.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:21 smithi026 ceph-mon[35496]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:21.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:21 smithi026 ceph-mon[35496]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:21.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:21 smithi003 ceph-mon[31476]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:21.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:21 smithi003 ceph-mon[31476]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:14:22.289 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:14:22.233+0000 7f98abac0640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T02:14:22.289 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:14:22.243+0000 7f98a78e8640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:14:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:14:22.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:14:22.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:22.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:22 smithi092 ceph-mon[36089]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:22.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:14:22.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:14:22.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:22.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:22 smithi003 ceph-mon[31476]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:22.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:14:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:14:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: from='osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:22 smithi026 ceph-mon[35496]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:14:22.773 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:14:22.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2024-09-12T02:14:23.053 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.425 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:23.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:23.426 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:23 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:23.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:23 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 -- 172.21.15.3:0/3372952231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4102660 msgr2=0x7fecf4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 --2- 172.21.15.3:0/3372952231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4102660 0x7fecf4104a50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009900 tx=0x7fecf002a410 comp rx=0 tx=0).stop 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 -- 172.21.15.3:0/3372952231 shutdown_connections 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 --2- 172.21.15.3:0/3372952231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 0x7fecf4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 --2- 172.21.15.3:0/3372952231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4104f90 0x7fecf4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 --2- 172.21.15.3:0/3372952231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4102660 0x7fecf4104a50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 -- 172.21.15.3:0/3372952231 >> 172.21.15.3:0/3372952231 conn(0x7fecf40fc280 msgr2=0x7fecf40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 -- 172.21.15.3:0/3372952231 shutdown_connections 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 -- 172.21.15.3:0/3372952231 wait complete. 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.511+0000 7fecfc277640 1 Processor -- start 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 -- start start 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4102660 0x7fecf4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 0x7fecf419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410d890 con 0x7fecf4104f90 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410d710 con 0x7fecf41078c0 2024-09-12T02:14:23.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfc277640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410da10 con 0x7fecf4102660 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfa013640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4102660 0x7fecf4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45022/0 (socket says 172.21.15.3:45022) 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 -- 172.21.15.3:0/2077794301 learned_addr learned my addr 172.21.15.3:0/2077794301 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecfa814640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 0x7fecf419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 -- 172.21.15.3:0/2077794301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4102660 msgr2=0x7fecf4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4102660 0x7fecf4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 -- 172.21.15.3:0/2077794301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 msgr2=0x7fecf419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 0x7fecf419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 -- 172.21.15.3:0/2077794301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf419fb50 con 0x7fecf4104f90 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.512+0000 7fecf9812640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fece400d740 tx=0x7fece400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.513+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4017070 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.513+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece400de50 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.513+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf419fe40 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.513+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4004e00 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.513+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41a02a0 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.514+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf40fdd70 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.515+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fece4004fa0 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.516+0000 7feceb7fe640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 0x7fece00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.516+0000 7fecfa013640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 0x7fece00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.516+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5862+0+0 (secure 0 0 0) 0x7fece40ce0f0 con 0x7fecf4104f90 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.516+0000 7fecfa013640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 0x7fece00a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fecf00314e0 tx=0x7fecf0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:23.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.520+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fece409b940 con 0x7fecf4104f90 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:23.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:23.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:23 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:23.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:23.706+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fecf4062630 con 0x7fece00a2bb0 2024-09-12T02:14:24.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:14:24.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:14:24.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:24.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi092 2024-09-12T02:14:24.453 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] boot 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:24.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:24.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi092 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] boot 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:24.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi092 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi092 to 5696M 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: osd.8 [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] boot 2024-09-12T02:14:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:14:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:14:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:24.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:25.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:25 smithi026 ceph-mon[35496]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:25.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:25 smithi026 ceph-mon[35496]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:14:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:25 smithi003 ceph-mon[31476]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:25.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:25 smithi003 ceph-mon[31476]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:14:25.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:25 smithi092 ceph-mon[36089]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:25.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:25 smithi092 ceph-mon[36089]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:14:26.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:26 smithi026 ceph-mon[35496]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:26 smithi026 ceph-mon[35496]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:14:26.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:26 smithi003 ceph-mon[31476]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:26.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:26 smithi003 ceph-mon[31476]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:14:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:26 smithi092 ceph-mon[36089]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:26 smithi092 ceph-mon[36089]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]: dispatch 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]': finished 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/581797910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:27.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:14:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]: dispatch 2024-09-12T02:14:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]': finished 2024-09-12T02:14:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:14:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:27.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/581797910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:27.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T02:14:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]: dispatch 2024-09-12T02:14:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4028040588' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7611642c-8229-45c0-b752-967b7260e2ae"}]': finished 2024-09-12T02:14:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:14:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/581797910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:28.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:28 smithi026 ceph-mon[35496]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:28.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:28 smithi003 ceph-mon[31476]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:28.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:28 smithi092 ceph-mon[36089]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:30.655 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:30 smithi003 ceph-mon[31476]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:30.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:30 smithi026 ceph-mon[35496]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:30.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:30 smithi092 ceph-mon[36089]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:32.588 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:32 smithi003 ceph-mon[31476]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:14:32.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:32 smithi026 ceph-mon[35496]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:14:32.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:32 smithi092 ceph-mon[36089]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:14:34.466 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:34 smithi003 ceph-mon[31476]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T02:14:34.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:34 smithi026 ceph-mon[35496]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T02:14:34.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:34 smithi092 ceph-mon[36089]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T02:14:35.496 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:14:35.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:35.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:14:35.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:35.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:14:35.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:36.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:36 smithi003 ceph-mon[31476]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:14:36.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:36 smithi003 ceph-mon[31476]: Deploying daemon osd.9 on smithi003 2024-09-12T02:14:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:36 smithi026 ceph-mon[35496]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:14:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:36 smithi026 ceph-mon[35496]: Deploying daemon osd.9 on smithi003 2024-09-12T02:14:36.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:36 smithi092 ceph-mon[36089]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-12T02:14:36.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:36 smithi092 ceph-mon[36089]: Deploying daemon osd.9 on smithi003 2024-09-12T02:14:38.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:38 smithi003 ceph-mon[31476]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:14:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:38 smithi026 ceph-mon[35496]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:14:38.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:38 smithi092 ceph-mon[36089]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:14:40.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:40 smithi003 ceph-mon[31476]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:40.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:40 smithi026 ceph-mon[35496]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:40.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:40 smithi092 ceph-mon[36089]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:41.588 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:41.588 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:41.588 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:41.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:42.607 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:42 smithi003 ceph-mon[31476]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:42.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:42 smithi026 ceph-mon[35496]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:42 smithi092 ceph-mon[36089]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:14:44.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.363+0000 7feceb7fe640 1 -- 172.21.15.3:0/2077794301 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fecf4062630 con 0x7fece00a2bb0 2024-09-12T02:14:44.364 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2024-09-12T02:14:44.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.365+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 msgr2=0x7fece00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.365+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 0x7fece00a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fecf00314e0 tx=0x7fecf0000bc0 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.365+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 msgr2=0x7fecf4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.365+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fece400d740 tx=0x7fece400dc00 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 shutdown_connections 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fece00a2bb0 0x7fece00a5070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecf41078c0 0x7fecf419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fecf4104f90 0x7fecf4198460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 --2- 172.21.15.3:0/2077794301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecf4102660 0x7fecf4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 >> 172.21.15.3:0/2077794301 conn(0x7fecf40fc280 msgr2=0x7fecf4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 shutdown_connections 2024-09-12T02:14:44.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:14:44.366+0000 7fecfc277640 1 -- 172.21.15.3:0/2077794301 wait complete. 2024-09-12T02:14:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:44.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:44.677 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:44 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:44 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:44 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:44.997 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T02:14:45.002 INFO:tasks.cephadm:Deploying osd.10 on smithi026 with /dev/vg_nvme/lv_1... 2024-09-12T02:14:45.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:14:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:14:45.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:45.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:45.665 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:45.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:45.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:45.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:14:45.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:45.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:46.508 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:46 smithi003 ceph-mon[31476]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:46.645 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:46 smithi026 ceph-mon[35496]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:46.690 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:46 smithi092 ceph-mon[36089]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:47.273 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:14:46 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:14:46.936+0000 7eff51816640 -1 osd.9 0 log_to_monitors true 2024-09-12T02:14:47.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:47 smithi026 ceph-mon[35496]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:14:47.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:47 smithi003 ceph-mon[31476]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:14:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:47 smithi092 ceph-mon[36089]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:14:47.872 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-12T02:14:47.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2024-09-12T02:14:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:14:48.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:48.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:14:48.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:14:48.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:14:48.557 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:48.558 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.641+0000 7fdca73ae640 1 -- 172.21.15.26:0/1856132643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0101d10 msgr2=0x7fdca0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.641+0000 7fdca73ae640 1 --2- 172.21.15.26:0/1856132643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0101d10 0x7fdca0102190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009900 tx=0x7fdc9402a410 comp rx=0 tx=0).stop 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- 172.21.15.26:0/1856132643 shutdown_connections 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- 172.21.15.26:0/1856132643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- 172.21.15.26:0/1856132643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0101d10 0x7fdca0102190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- 172.21.15.26:0/1856132643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0100b10 0x7fdca0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- 172.21.15.26:0/1856132643 >> 172.21.15.26:0/1856132643 conn(0x7fdca00fc2a0 msgr2=0x7fdca00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- 172.21.15.26:0/1856132643 shutdown_connections 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- 172.21.15.26:0/1856132643 wait complete. 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 Processor -- start 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- start start 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 0x7fdca019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010c790 con 0x7fdca0101d10 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010c610 con 0x7fdca0100b10 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca73ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdca010c910 con 0x7fdca01026d0 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.642+0000 7fdca514a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca514a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55094/0 (socket says 172.21.15.26:55094) 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca514a640 1 -- 172.21.15.26:0/2565556546 learned_addr learned my addr 172.21.15.26:0/2565556546 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 0x7fdca019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca594b640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 -- 172.21.15.26:0/2565556546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 msgr2=0x7fdca019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 -- 172.21.15.26:0/2565556546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 msgr2=0x7fdca006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:14:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 -- 172.21.15.26:0/2565556546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca019fb70 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca514a640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca594b640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca4949640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 0x7fdca019cc40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc94031440 tx=0x7fdc94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9403f070 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca019fda0 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc94003cd0 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94035070 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.643+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a01f0 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.644+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca0107f60 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.646+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdc9403b4f0 con 0x7fdca0101d10 2024-09-12T02:14:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.646+0000 7fdc9e7fc640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 0x7fdc880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:14:48.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.647+0000 7fdca514a640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 0x7fdc880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:14:48.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.647+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7fdc940ef0d0 con 0x7fdca0101d10 2024-09-12T02:14:48.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.647+0000 7fdca514a640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 0x7fdc880a50c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc980058c0 tx=0x7fdc98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:14:48.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.650+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc940bc260 con 0x7fdca0101d10 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:48.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:14:48.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:48.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:14:48.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:14:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:14:48.835+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdca0062630 con 0x7fdc880a2c00 2024-09-12T02:14:49.278 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:14:48.847+0000 7eff4d799640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T02:14:49.278 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:14:48 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:14:48.857+0000 7eff455b9640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi003 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:49.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:49.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:49.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:49 smithi026 ceph-mon[35496]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi003 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:49.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:49 smithi003 ceph-mon[31476]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi003 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi003 to 4273M 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:14:49.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:49 smithi092 ceph-mon[36089]: from='osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345]' entity='osd.9' 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] boot 2024-09-12T02:14:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:14:50.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] boot 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:14:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: from='client.14463 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: osd.9 [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] boot 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:14:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:14:52.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:52.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:14:52.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/4256176047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]': finished 2024-09-12T02:14:52.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:14:52.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:14:52.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:51 smithi026 ceph-mon[35496]: from='client.? 172.21.15.26:0/1023319990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:52.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/4256176047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]': finished 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:14:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:51 smithi003 ceph-mon[31476]: from='client.? 172.21.15.26:0/1023319990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:52.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/4256176047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]: dispatch 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ab8f17b-02b0-441a-bddf-77679403a160"}]': finished 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:14:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:51 smithi092 ceph-mon[36089]: from='client.? 172.21.15.26:0/1023319990' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:14:54.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:53 smithi026 ceph-mon[35496]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:54.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:53 smithi003 ceph-mon[31476]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:54.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:53 smithi092 ceph-mon[36089]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:56.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:55 smithi026 ceph-mon[35496]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:56.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:55 smithi003 ceph-mon[31476]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:56.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:55 smithi092 ceph-mon[36089]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:58.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:57 smithi026 ceph-mon[35496]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:58.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:57 smithi003 ceph-mon[31476]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:14:58.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:57 smithi092 ceph-mon[36089]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:00.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:14:59 smithi026 ceph-mon[35496]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:14:59 smithi003 ceph-mon[31476]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:00.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:14:59 smithi092 ceph-mon[36089]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:01.006 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:15:01.006 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:01.006 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:00 smithi026 ceph-mon[35496]: Deploying daemon osd.10 on smithi026 2024-09-12T02:15:01.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:15:01.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:01.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:00 smithi003 ceph-mon[31476]: Deploying daemon osd.10 on smithi026 2024-09-12T02:15:01.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:15:01.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:01.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:00 smithi092 ceph-mon[36089]: Deploying daemon osd.10 on smithi026 2024-09-12T02:15:01.957 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:01 smithi026 ceph-mon[35496]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:01 smithi003 ceph-mon[31476]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:02.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:01 smithi092 ceph-mon[36089]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:04.203 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:03 smithi026 ceph-mon[35496]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:04.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:03 smithi003 ceph-mon[31476]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:04.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:03 smithi092 ceph-mon[36089]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:06.164 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:05 smithi026 ceph-mon[35496]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:06.164 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:05 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:06.164 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:05 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:06.164 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:05 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:06.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:05 smithi003 ceph-mon[31476]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:05 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:06.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:05 smithi092 ceph-mon[36089]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:06.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:05 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:05 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:05 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:07 smithi026 ceph-mon[35496]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:08.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:07 smithi003 ceph-mon[31476]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:08.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:07 smithi092 ceph-mon[36089]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:09.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.334+0000 7fdc9e7fc640 1 -- 172.21.15.26:0/2565556546 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdca0062630 con 0x7fdc880a2c00 2024-09-12T02:15:09.335 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 10 on host 'smithi026' 2024-09-12T02:15:09.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 msgr2=0x7fdc880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:09.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 0x7fdc880a50c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdc980058c0 tx=0x7fdc98005f90 comp rx=0 tx=0).stop 2024-09-12T02:15:09.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 msgr2=0x7fdca019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:09.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 0x7fdca019cc40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc94031440 tx=0x7fdc94004270 comp rx=0 tx=0).stop 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 shutdown_connections 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdc880a2c00 0x7fdc880a50c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdca01026d0 0x7fdca019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdca0101d10 0x7fdca019cc40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 --2- 172.21.15.26:0/2565556546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdca0100b10 0x7fdca006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 >> 172.21.15.26:0/2565556546 conn(0x7fdca00fc2a0 msgr2=0x7fdca0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 shutdown_connections 2024-09-12T02:15:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-12T02:15:09.337+0000 7fdca73ae640 1 -- 172.21.15.26:0/2565556546 wait complete. 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:09.962 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:09.989 DEBUG:teuthology.orchestra.run.smithi026:osd.10> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T02:15:09.991 INFO:tasks.cephadm:Deploying osd.11 on smithi092 with /dev/vg_nvme/lv_1... 2024-09-12T02:15:09.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:15:10.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:10.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.019 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:10.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:10.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:10.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.020 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:10.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:15:11.212 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:15:10 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:15:10.847+0000 7f09c6c6c640 -1 osd.10 0 log_to_monitors true 2024-09-12T02:15:11.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:11 smithi092 ceph-mon[36089]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:11.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:11 smithi092 ceph-mon[36089]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:11.698 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:11 smithi092 ceph-mon[36089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:12.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:11 smithi026 ceph-mon[35496]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:11 smithi026 ceph-mon[35496]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:11 smithi026 ceph-mon[35496]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:12.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:11 smithi003 ceph-mon[31476]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:12.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:11 smithi003 ceph-mon[31476]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:12.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:11 smithi003 ceph-mon[31476]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:15:12.886 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-mon[35496]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:15:12.887 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-mon[35496]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:15:12.887 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:12.887 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-mon[35496]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:12.887 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-mon[35496]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:12.887 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:15:12.601+0000 7f09c2bef640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T02:15:12.887 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:15:12 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:15:12.611+0000 7f09bea17640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:15:13.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:12 smithi003 ceph-mon[31476]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:15:13.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:12 smithi003 ceph-mon[31476]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:15:13.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:12 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:13.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:12 smithi003 ceph-mon[31476]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:13.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:12 smithi003 ceph-mon[31476]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:13.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:12 smithi092 ceph-mon[36089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:15:13.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:12 smithi092 ceph-mon[36089]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:15:13.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:12 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:13.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:12 smithi092 ceph-mon[36089]: from='osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:13.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:12 smithi092 ceph-mon[36089]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2024-09-12T02:15:13.074 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-09-12T02:15:13.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-09-12T02:15:13.397 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi026 2024-09-12T02:15:13.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:13.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:13.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:13.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:13.885 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- 172.21.15.92:0/1821090861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc102690 msgr2=0x7f28cc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/1821090861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc102690 0x7f28cc108f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009ca0 tx=0x7f28c402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- 172.21.15.92:0/1821090861 shutdown_connections 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/1821090861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc102690 0x7f28cc108f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/1821090861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc101cd0 0x7f28cc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/1821090861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 0x7f28cc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- 172.21.15.92:0/1821090861 >> 172.21.15.92:0/1821090861 conn(0x7f28cc0fc280 msgr2=0x7f28cc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- 172.21.15.92:0/1821090861 shutdown_connections 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- 172.21.15.92:0/1821090861 wait complete. 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 Processor -- start 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.878+0000 7f28d3c3a640 1 -- start start 2024-09-12T02:15:13.886 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 0x7f28cc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc102690 0x7f28cc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc10c730 con 0x7f28cc100ad0 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc10c5b0 con 0x7f28cc102690 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc10c8b0 con 0x7f28cc101cd0 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58374/0 (socket says 172.21.15.92:58374) 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 -- 172.21.15.92:0/2660095590 learned_addr learned my addr 172.21.15.92:0/2660095590 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d19d6640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 0x7f28cc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d21d7640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc102690 0x7f28cc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:13.887 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 -- 172.21.15.92:0/2660095590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc102690 msgr2=0x7f28cc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc102690 0x7f28cc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 -- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 msgr2=0x7f28cc197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 0x7f28cc197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 -- 172.21.15.92:0/2660095590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc19fa40 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d11d5640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c920 tx=0x7f28c800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007c70 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c8007e10 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc19fd30 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8018890 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.879+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1a0260 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.880+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc1a04f0 con 0x7f28cc101cd0 2024-09-12T02:15:13.888 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.881+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f28c8018a30 con 0x7f28cc101cd0 2024-09-12T02:15:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.882+0000 7f28c2ffd640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 0x7f28b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.882+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6727+0+0 (secure 0 0 0) 0x7f28c80ceb50 con 0x7f28cc101cd0 2024-09-12T02:15:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.882+0000 7f28d19d6640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 0x7f28b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.883+0000 7f28d19d6640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 0x7f28b80a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005db0 tx=0x7f28bc005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:13.889 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:13.886+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28c8014030 con 0x7f28cc101cd0 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi026 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:14.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi026 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi026 to 4273M 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:14.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:14.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:14.060+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f28cc06a680 con 0x7f28b80a2c30 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] boot 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:15:14.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:15:14.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:14 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] boot 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:15:15.062 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:14 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: osd.10 [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] boot 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:15:15.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:15:15.064 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:14 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:15.724 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:15 smithi092 ceph-mon[36089]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:15.724 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:15 smithi092 ceph-mon[36089]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:15:15.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:15 smithi003 ceph-mon[31476]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:15.864 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:15 smithi003 ceph-mon[31476]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:15:16.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:15 smithi026 ceph-mon[35496]: pgmap v227: 1 pgs: 1 peering; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:15:16.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:15 smithi026 ceph-mon[35496]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:15:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:15:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3949918345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]': finished 2024-09-12T02:15:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:15:16.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:16 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3949918345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]': finished 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:15:17.017 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:16 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:17.033 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T02:15:17.034 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3949918345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:17.034 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]: dispatch 2024-09-12T02:15:17.034 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84dbc141-f1cc-4f68-909e-53f67c8e5190"}]': finished 2024-09-12T02:15:17.034 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:15:17.034 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:16 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:17.916 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:17 smithi026 ceph-mon[35496]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:17.916 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:17 smithi026 ceph-mon[35496]: from='client.? 172.21.15.92:0/3289842398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:15:17.916 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:17 smithi026 ceph-mon[35496]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:15:17.916 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:17 smithi003 ceph-mon[31476]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:17 smithi003 ceph-mon[31476]: from='client.? 172.21.15.92:0/3289842398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:15:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:17 smithi003 ceph-mon[31476]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:15:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:18.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:17 smithi092 ceph-mon[36089]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:18.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:17 smithi092 ceph-mon[36089]: from='client.? 172.21.15.92:0/3289842398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:15:18.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:17 smithi092 ceph-mon[36089]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:15:18.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:20.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:19 smithi026 ceph-mon[35496]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:20.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:19 smithi003 ceph-mon[31476]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:20.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:19 smithi092 ceph-mon[36089]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:22.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:21 smithi026 ceph-mon[35496]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T02:15:22.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:21 smithi003 ceph-mon[31476]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T02:15:22.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:21 smithi092 ceph-mon[36089]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-12T02:15:24.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:23 smithi026 ceph-mon[35496]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T02:15:24.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:23 smithi003 ceph-mon[31476]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T02:15:24.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:23 smithi092 ceph-mon[36089]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-12T02:15:25.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:25 smithi092 ceph-mon[36089]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:15:25.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:15:25.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:26.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:25 smithi026 ceph-mon[35496]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:15:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:15:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:25 smithi003 ceph-mon[31476]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:15:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:15:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:26.934 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:26 smithi092 ceph-mon[36089]: Deploying daemon osd.11 on smithi092 2024-09-12T02:15:27.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:26 smithi026 ceph-mon[35496]: Deploying daemon osd.11 on smithi092 2024-09-12T02:15:27.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:26 smithi003 ceph-mon[31476]: Deploying daemon osd.11 on smithi092 2024-09-12T02:15:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:27 smithi092 ceph-mon[36089]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:15:28.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:27 smithi026 ceph-mon[35496]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:15:28.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:27 smithi003 ceph-mon[31476]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:15:30.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:29 smithi026 ceph-mon[35496]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T02:15:30.024 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:29 smithi092 ceph-mon[36089]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T02:15:30.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:29 smithi003 ceph-mon[31476]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-12T02:15:31.985 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:31 smithi092 ceph-mon[36089]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:15:31.985 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:31.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:31.986 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:31 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:32.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:31 smithi026 ceph-mon[35496]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:15:32.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:32.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:32.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:31 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:32.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:31 smithi003 ceph-mon[31476]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:15:32.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:32.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:32.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:31 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:34.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:33 smithi026 ceph-mon[35496]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:34.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:33 smithi003 ceph-mon[31476]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:34.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:33 smithi092 ceph-mon[36089]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:35.151 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.150+0000 7f28c2ffd640 1 -- 172.21.15.92:0/2660095590 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f28cc06a680 con 0x7f28b80a2c30 2024-09-12T02:15:35.151 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 11 on host 'smithi092' 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.152+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 msgr2=0x7f28b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.152+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 0x7f28b80a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005db0 tx=0x7f28bc005d40 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.152+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 msgr2=0x7f28cc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.152+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c920 tx=0x7f28c800cde0 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 shutdown_connections 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28b80a2c30 0x7f28b80a50f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28cc102690 0x7f28cc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28cc101cd0 0x7f28cc198350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 --2- 172.21.15.92:0/2660095590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28cc100ad0 0x7f28cc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 >> 172.21.15.92:0/2660095590 conn(0x7f28cc0fc280 msgr2=0x7f28cc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 shutdown_connections 2024-09-12T02:15:35.154 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:35.153+0000 7f28d3c3a640 1 -- 172.21.15.92:0/2660095590 wait complete. 2024-09-12T02:15:35.832 DEBUG:teuthology.orchestra.run.smithi092:osd.11> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T02:15:35.834 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T02:15:35.835 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:15:36.121 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:36.401 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:36 smithi003 ceph-mon[31476]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:36.401 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.402 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.402 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.402 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-mon[36089]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:36.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.467 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.468 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:36 smithi026 ceph-mon[35496]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.596+0000 7fd4af3ae640 1 -- 172.21.15.3:0/734807166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 msgr2=0x7fd4a8101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.596+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/734807166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a8101be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0009ca0 tx=0x7fd4a002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 -- 172.21.15.3:0/734807166 shutdown_connections 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/734807166 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a8101be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/734807166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/734807166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 -- 172.21.15.3:0/734807166 >> 172.21.15.3:0/734807166 conn(0x7fd4a80fc280 msgr2=0x7fd4a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 -- 172.21.15.3:0/734807166 shutdown_connections 2024-09-12T02:15:36.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 -- 172.21.15.3:0/734807166 wait complete. 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 Processor -- start 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.597+0000 7fd4af3ae640 1 -- start start 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810fa40 con 0x7fd4a8101730 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810f8c0 con 0x7fd4a8100d90 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4af3ae640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a810fbc0 con 0x7fd4a8100450 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad94b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:36.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad94b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a81a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39050/0 (socket says 172.21.15.3:39050) 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad94b640 1 -- 172.21.15.3:0/3470766005 learned_addr learned my addr 172.21.15.3:0/3470766005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ac949640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 -- 172.21.15.3:0/3470766005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 msgr2=0x7fd4a819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 -- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 msgr2=0x7fd4a81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 -- 172.21.15.3:0/3470766005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a81a1c90 con 0x7fd4a8100450 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ac949640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.598+0000 7fd4ad14a640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a819a060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400e9f0 tx=0x7fd4a400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.599+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400ccc0 con 0x7fd4a8100450 2024-09-12T02:15:36.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.599+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4a4005020 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.599+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a81a1f80 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.599+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40049e0 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.599+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81a2370 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.600+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a810b1a0 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.601+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd4a4005560 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.601+0000 7fd49e7fc640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 0x7fd4940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.602+0000 7fd4ac949640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 0x7fd4940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.602+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7fd4a40cead0 con 0x7fd4a8100450 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.602+0000 7fd4ac949640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 0x7fd4940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4a819b580 tx=0x7fd49800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:36.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.606+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4a40051d0 con 0x7fd4a8100450 2024-09-12T02:15:36.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.784+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd4a8062e20 con 0x7fd4a8100450 2024-09-12T02:15:36.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.785+0000 7fd49e7fc640 1 -- 172.21.15.3:0/3470766005 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fd4a409bf70 con 0x7fd4a8100450 2024-09-12T02:15:36.786 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 msgr2=0x7fd4940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 0x7fd4940a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4a819b580 tx=0x7fd49800b5f0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 msgr2=0x7fd4a819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a819a060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400e9f0 tx=0x7fd4a400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 shutdown_connections 2024-09-12T02:15:36.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd4940a2bb0 0x7fd4940a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd4a8101730 0x7fd4a81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4a8100d90 0x7fd4a819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 --2- 172.21.15.3:0/3470766005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4a8100450 0x7fd4a819a060 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 >> 172.21.15.3:0/3470766005 conn(0x7fd4a80fc280 msgr2=0x7fd4a80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 shutdown_connections 2024-09-12T02:15:36.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:36.787+0000 7fd4af3ae640 1 -- 172.21.15.3:0/3470766005 wait complete. 2024-09-12T02:15:37.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:37 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3470766005' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:37.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:37 smithi092 ceph-mon[36089]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:37 smithi092 ceph-mon[36089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:15:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:15:36.957+0000 7f2e205f2640 -1 osd.11 0 log_to_monitors true 2024-09-12T02:15:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:37 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3470766005' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:37 smithi026 ceph-mon[35496]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:37 smithi026 ceph-mon[35496]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:37 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3470766005' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:37.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:37 smithi003 ceph-mon[31476]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:37 smithi003 ceph-mon[31476]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:15:37.572 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726107313,"num_in_osds":12,"osd_in_since":1726107316,"num_remapped_pgs":0} 2024-09-12T02:15:38.437 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:15:38.155+0000 7f2e1c575640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T02:15:38.437 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:15:38.166+0000 7f2e1839d640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:15:38.437 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:38.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:15:38.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:15:38.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:38.438 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:38 smithi092 ceph-mon[36089]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:38 smithi026 ceph-mon[35496]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:15:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: from='osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:15:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:38 smithi003 ceph-mon[31476]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-09-12T02:15:38.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:15:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:39 smithi003 ceph-mon[31476]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:15:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:39 smithi003 ceph-mon[31476]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:15:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:39.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 -- 172.21.15.3:0/403527116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 msgr2=0x7f366c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:39.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 --2- 172.21.15.3:0/403527116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c100ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3668009900 tx=0x7f366802a410 comp rx=0 tx=0).stop 2024-09-12T02:15:39.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 -- 172.21.15.3:0/403527116 shutdown_connections 2024-09-12T02:15:39.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 --2- 172.21.15.3:0/403527116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 --2- 172.21.15.3:0/403527116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 0x7f366c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 --2- 172.21.15.3:0/403527116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c100ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 -- 172.21.15.3:0/403527116 >> 172.21.15.3:0/403527116 conn(0x7f366c0fc280 msgr2=0x7f366c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 -- 172.21.15.3:0/403527116 shutdown_connections 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.333+0000 7f3673b68640 1 -- 172.21.15.3:0/403527116 wait complete. 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 Processor -- start 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 -- start start 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 0x7f366c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10c7b0 con 0x7f366c100ad0 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10c630 con 0x7f366c101cd0 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3673b68640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f366c10c930 con 0x7f366c102690 2024-09-12T02:15:39.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3671904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3671904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34276/0 (socket says 172.21.15.3:34276) 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3671904640 1 -- 172.21.15.3:0/4050682147 learned_addr learned my addr 172.21.15.3:0/4050682147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3672105640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.334+0000 7f3671103640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 0x7f366c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 -- 172.21.15.3:0/4050682147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 msgr2=0x7f366c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 0x7f366c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 -- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 msgr2=0x7f366c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 -- 172.21.15.3:0/4050682147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c19fb60 con 0x7f366c102690 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3671904640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3672105640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c19f4f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3664009fd0 tx=0x7f366400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366400ce20 con 0x7f366c102690 2024-09-12T02:15:39.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3664005020 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c19fe50 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664010720 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.335+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a0350 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.336+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c1a0610 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.337+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3664005540 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.338+0000 7f3662ffd640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 0x7f36580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.338+0000 7f3671904640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 0x7f36580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.338+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f36640cf200 con 0x7f366c102690 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.338+0000 7f3671904640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 0x7f36580a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3668000c00 tx=0x7f36680315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:39.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.342+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f366409c560 con 0x7f366c102690 2024-09-12T02:15:39.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:39 smithi026 ceph-mon[35496]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:15:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:39 smithi026 ceph-mon[35496]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:15:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.517+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f366c1a0880 con 0x7f366c102690 2024-09-12T02:15:39.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.518+0000 7f3662ffd640 1 -- 172.21.15.3:0/4050682147 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f366409be60 con 0x7f366c102690 2024-09-12T02:15:39.519 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 msgr2=0x7f36580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 0x7f36580a50f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3668000c00 tx=0x7f36680315b0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 msgr2=0x7f366c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:39.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c19f4f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3664009fd0 tx=0x7f366400ef10 comp rx=0 tx=0).stop 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 shutdown_connections 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f36580a2c30 0x7f36580a50f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f366c102690 0x7f366c19f4f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f366c101cd0 0x7f366c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 --2- 172.21.15.3:0/4050682147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f366c100ad0 0x7f366c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 >> 172.21.15.3:0/4050682147 conn(0x7f366c0fc280 msgr2=0x7f366c06a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 shutdown_connections 2024-09-12T02:15:39.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:39.520+0000 7f3673b68640 1 -- 172.21.15.3:0/4050682147 wait complete. 2024-09-12T02:15:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:39 smithi092 ceph-mon[36089]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-09-12T02:15:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:39 smithi092 ceph-mon[36089]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:15:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:40.157 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726107339,"num_in_osds":12,"osd_in_since":1726107316,"num_remapped_pgs":0} 2024-09-12T02:15:40.157 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: purged_snaps scrub starts 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: purged_snaps scrub ok 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] boot 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: Detected new or changed devices on smithi092 2024-09-12T02:15:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:40 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4050682147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: purged_snaps scrub starts 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: purged_snaps scrub ok 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] boot 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: Detected new or changed devices on smithi092 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.517 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.517 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:40 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4050682147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: purged_snaps scrub starts 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: purged_snaps scrub ok 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: osd.11 [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] boot 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: Detected new or changed devices on smithi092 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi092 to 4272M 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:40.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:40 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4050682147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:15:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 -- 172.21.15.3:0/1292750952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec102790 msgr2=0x7f97ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 --2- 172.21.15.3:0/1292750952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec102790 0x7f97ec1067e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e002a410 comp rx=0 tx=0).stop 2024-09-12T02:15:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 -- 172.21.15.3:0/1292750952 shutdown_connections 2024-09-12T02:15:40.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 --2- 172.21.15.3:0/1292750952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec106f10 0x7f97ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 --2- 172.21.15.3:0/1292750952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec102790 0x7f97ec1067e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 --2- 172.21.15.3:0/1292750952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec101de0 0x7f97ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 -- 172.21.15.3:0/1292750952 >> 172.21.15.3:0/1292750952 conn(0x7f97ec0fd8c0 msgr2=0x7f97ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.898+0000 7f97f0b24640 1 -- 172.21.15.3:0/1292750952 shutdown_connections 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.899+0000 7f97f0b24640 1 -- 172.21.15.3:0/1292750952 wait complete. 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.899+0000 7f97f0b24640 1 Processor -- start 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 -- start start 2024-09-12T02:15:40.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 0x7f97ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec102790 0x7f97ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec111f70 con 0x7f97ec106f10 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec111df0 con 0x7f97ec101de0 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97f0b24640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1120f0 con 0x7f97ec102790 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34286/0 (socket says 172.21.15.3:34286) 2024-09-12T02:15:40.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97ead9d640 1 -- 172.21.15.3:0/2871357368 learned_addr learned my addr 172.21.15.3:0/2871357368 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.900+0000 7f97e9d9b640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec102790 0x7f97ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ea59c640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 0x7f97ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 -- 172.21.15.3:0/2871357368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec102790 msgr2=0x7f97ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec102790 0x7f97ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 -- 172.21.15.3:0/2871357368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 msgr2=0x7f97ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 0x7f97ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 -- 172.21.15.3:0/2871357368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec1a6950 con 0x7f97ec106f10 2024-09-12T02:15:40.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ea59c640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 0x7f97ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97ead9d640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009fd0 tx=0x7f97e400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4017070 con 0x7f97ec106f10 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97e4003e50 con 0x7f97ec106f10 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec1a6c40 con 0x7f97ec106f10 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40106c0 con 0x7f97ec106f10 2024-09-12T02:15:40.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.901+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec1a7100 con 0x7f97ec106f10 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.902+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ec1021c0 con 0x7f97ec106f10 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.903+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f97e4003970 con 0x7f97ec106f10 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.904+0000 7f97db7fe640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 0x7f97d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.904+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f97e40ce8d0 con 0x7f97ec106f10 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.904+0000 7f97ea59c640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 0x7f97d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:40.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.905+0000 7f97ea59c640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 0x7f97d40a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0058c0 tx=0x7f97dc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:40.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:40.908+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97e409bcd0 con 0x7f97ec106f10 2024-09-12T02:15:41.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.085+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97ec10a150 con 0x7f97ec106f10 2024-09-12T02:15:41.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.087+0000 7f97db7fe640 1 -- 172.21.15.3:0/2871357368 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18290 (secure 0 0 0) 0x7f97e409b5d0 con 0x7f97ec106f10 2024-09-12T02:15:41.087 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:41.088 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":76,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:15:40.311580+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:15:41.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 msgr2=0x7f97d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:41.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 0x7f97d40a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0058c0 tx=0x7f97dc005850 comp rx=0 tx=0).stop 2024-09-12T02:15:41.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 msgr2=0x7f97ec074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009fd0 tx=0x7f97e400b260 comp rx=0 tx=0).stop 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 shutdown_connections 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97d40a2b80 0x7f97d40a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f97ec106f10 0x7f97ec074f70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97ec102790 0x7f97ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 --2- 172.21.15.3:0/2871357368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97ec101de0 0x7f97ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 >> 172.21.15.3:0/2871357368 conn(0x7f97ec0fd8c0 msgr2=0x7f97ec0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 shutdown_connections 2024-09-12T02:15:41.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:41.089+0000 7f97f0b24640 1 -- 172.21.15.3:0/2871357368 wait complete. 2024-09-12T02:15:41.394 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:41 smithi003 ceph-mon[31476]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:15:41.395 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:41 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2871357368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:15:41.760 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:11:52.858011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:15:41.760 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T02:15:41.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:41 smithi026 ceph-mon[35496]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:15:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:41 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2871357368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:15:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:41 smithi092 ceph-mon[36089]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:15:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:41 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2871357368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:15:42.012 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:42.392 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:42 smithi003 ceph-mon[31476]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:42.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- 172.21.15.3:0/1317166381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc1036e0 msgr2=0x7f2cfc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 --2- 172.21.15.3:0/1317166381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc107b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009930 tx=0x7f2cf002a420 comp rx=0 tx=0).stop 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- 172.21.15.3:0/1317166381 shutdown_connections 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 --2- 172.21.15.3:0/1317166381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1082a0 0x7f2cfc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 --2- 172.21.15.3:0/1317166381 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc107b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 --2- 172.21.15.3:0/1317166381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc102d30 0x7f2cfc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- 172.21.15.3:0/1317166381 >> 172.21.15.3:0/1317166381 conn(0x7f2cfc0fc7d0 msgr2=0x7f2cfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- 172.21.15.3:0/1317166381 shutdown_connections 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- 172.21.15.3:0/1317166381 wait complete. 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 Processor -- start 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.477+0000 7f2d0309f640 1 -- start start 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:42.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc1082a0 0x7f2cfc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc1120c0 con 0x7f2cfc102d30 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc111f40 con 0x7f2cfc1082a0 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc112240 con 0x7f2cfc1036e0 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34308/0 (socket says 172.21.15.3:34308) 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 -- 172.21.15.3:0/2456544492 learned_addr learned my addr 172.21.15.3:0/2456544492 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2cfbfff640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0163c640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc1082a0 0x7f2cfc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 -- 172.21.15.3:0/2456544492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 msgr2=0x7f2cfc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 -- 172.21.15.3:0/2456544492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc1082a0 msgr2=0x7f2cfc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc1082a0 0x7f2cfc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 -- 172.21.15.3:0/2456544492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc1a6a90 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2cfbfff640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d00e3b640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00b4a0 tx=0x7f2cec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec014070 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc1a6d80 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cec010af0 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.478+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec017d10 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.479+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a71e0 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.480+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc06a850 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.481+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2cec017070 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.481+0000 7f2cf9ffb640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 0x7f2ce40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.482+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2cec0cdfb0 con 0x7f2cfc102d30 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.482+0000 7f2cfbfff640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 0x7f2ce40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.482+0000 7f2cfbfff640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 0x7f2ce40a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009900 tx=0x7f2cf0031690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.486+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cec09b3b0 con 0x7f2cfc102d30 2024-09-12T02:15:42.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.659+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cfc10a2b0 con 0x7f2cfc102d30 2024-09-12T02:15:42.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.660+0000 7f2cf9ffb640 1 -- 172.21.15.3:0/2456544492 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f2cec09acb0 con 0x7f2cfc102d30 2024-09-12T02:15:42.660 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T02:15:42.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 msgr2=0x7f2ce40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 0x7f2ce40a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009900 tx=0x7f2cf0031690 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 msgr2=0x7f2cfc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00b4a0 tx=0x7f2cec00b960 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 shutdown_connections 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2ce40a2b80 0x7f2ce40a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cfc1082a0 0x7f2cfc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cfc1036e0 0x7f2cfc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 --2- 172.21.15.3:0/2456544492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2cfc102d30 0x7f2cfc100990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.662+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 >> 172.21.15.3:0/2456544492 conn(0x7f2cfc0fc7d0 msgr2=0x7f2cfc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.663+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 shutdown_connections 2024-09-12T02:15:42.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:42.663+0000 7f2d0309f640 1 -- 172.21.15.3:0/2456544492 wait complete. 2024-09-12T02:15:42.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:42 smithi026 ceph-mon[35496]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:42.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:42 smithi092 ceph-mon[36089]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.a on smithi003 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.d on smithi003 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.g on smithi003 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.j on smithi003 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.b on smithi026 2024-09-12T02:15:43.446 INFO:tasks.cephadm:Adding mds.e on smithi026 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.h on smithi026 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.k on smithi026 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.c on smithi092 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.f on smithi092 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.i on smithi092 2024-09-12T02:15:43.447 INFO:tasks.cephadm:Adding mds.l on smithi092 2024-09-12T02:15:43.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l' 2024-09-12T02:15:43.702 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:15:43.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:43 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2456544492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:15:43.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:43 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2456544492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:15:43.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:43 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2456544492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 -- 172.21.15.92:0/846067254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 msgr2=0x7efd1c101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.231 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 --2- 172.21.15.92:0/846067254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c101fe0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd10009900 tx=0x7efd1002a410 comp rx=0 tx=0).stop 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 -- 172.21.15.92:0/846067254 shutdown_connections 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 --2- 172.21.15.92:0/846067254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 0x7efd1c108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 --2- 172.21.15.92:0/846067254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c101fe0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 --2- 172.21.15.92:0/846067254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 0x7efd1c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 -- 172.21.15.92:0/846067254 >> 172.21.15.92:0/846067254 conn(0x7efd1c0fc0f0 msgr2=0x7efd1c0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 -- 172.21.15.92:0/846067254 shutdown_connections 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.215+0000 7efd21804640 1 -- 172.21.15.92:0/846067254 wait complete. 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 Processor -- start 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 -- start start 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 0x7efd1c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 0x7efd1c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c5e0 con 0x7efd1c102520 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c460 con 0x7efd1c100960 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd21804640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c760 con 0x7efd1c101b60 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd1a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd1a7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35530/0 (socket says 172.21.15.92:35530) 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd1a7fc640 1 -- 172.21.15.92:0/3272131754 learned_addr learned my addr 172.21.15.92:0/3272131754 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.216+0000 7efd1b7fe640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 0x7efd1c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1affd640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 0x7efd1c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 -- 172.21.15.92:0/3272131754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 msgr2=0x7efd1c196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 0x7efd1c196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 -- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 msgr2=0x7efd1c19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 0x7efd1c19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 -- 172.21.15.92:0/3272131754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c19e880 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd1a7fc640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd10009230 tx=0x7efd10004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1003f030 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c19eb10 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd10003ce0 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10035070 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.217+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c19eff0 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.218+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c107db0 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.219+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efd10042030 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.220+0000 7efd20802640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 0x7efd040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.220+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7efd100ef580 con 0x7efd1c101b60 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.220+0000 7efd1affd640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 0x7efd040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.221+0000 7efd1affd640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 0x7efd040a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:44.235 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.224+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd100bc980 con 0x7efd1c101b60 2024-09-12T02:15:44.416 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:44 smithi092 ceph-mon[36089]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.429+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l", "target": ["mon-mgr", ""]}) -- 0x7efd1c062730 con 0x7efd040a2b80 2024-09-12T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.437+0000 7efd20802640 1 -- 172.21.15.92:0/3272131754 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7efd1c062730 con 0x7efd040a2b80 2024-09-12T02:15:44.437 INFO:teuthology.orchestra.run.smithi092.stdout:Scheduled mds.all update... 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 msgr2=0x7efd040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 0x7efd040a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c005850 comp rx=0 tx=0).stop 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 msgr2=0x7efd1c1971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd10009230 tx=0x7efd10004360 comp rx=0 tx=0).stop 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 shutdown_connections 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efd040a2b80 0x7efd040a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.439 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd1c102520 0x7efd1c19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd1c101b60 0x7efd1c1971a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 --2- 172.21.15.92:0/3272131754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd1c100960 0x7efd1c196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 >> 172.21.15.92:0/3272131754 conn(0x7efd1c0fc0f0 msgr2=0x7efd1c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 shutdown_connections 2024-09-12T02:15:44.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-09-12T02:15:44.439+0000 7efd21804640 1 -- 172.21.15.92:0/3272131754 wait complete. 2024-09-12T02:15:44.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:44 smithi026 ceph-mon[35496]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:44.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:44 smithi003 ceph-mon[31476]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:45.165 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T02:15:45.168 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T02:15:45.211 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T02:15:45.254 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T02:15:45.297 DEBUG:teuthology.orchestra.run.smithi026:mds.b> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T02:15:45.299 DEBUG:teuthology.orchestra.run.smithi026:mds.e> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T02:15:45.345 DEBUG:teuthology.orchestra.run.smithi026:mds.h> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T02:15:45.388 DEBUG:teuthology.orchestra.run.smithi026:mds.k> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T02:15:45.430 DEBUG:teuthology.orchestra.run.smithi092:mds.c> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T02:15:45.433 DEBUG:teuthology.orchestra.run.smithi092:mds.f> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T02:15:45.476 DEBUG:teuthology.orchestra.run.smithi092:mds.i> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T02:15:45.519 DEBUG:teuthology.orchestra.run.smithi092:mds.l> sudo journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T02:15:45.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:45.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:45.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l;count:12 2024-09-12T02:15:45.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:45 smithi003 ceph-mon[31476]: Deploying daemon mds.l on smithi092 2024-09-12T02:15:45.562 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T02:15:45.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:45.656 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:45.656 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l;count:12 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.657 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:45 smithi092 ceph-mon[36089]: Deploying daemon mds.l on smithi092 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l;count:12 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:45.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:45.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:45.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:45.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:45 smithi026 ceph-mon[35496]: Deploying daemon mds.l on smithi092 2024-09-12T02:15:45.821 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:46.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 -- 172.21.15.3:0/2698463129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20102790 msgr2=0x7fae201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 --2- 172.21.15.3:0/2698463129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20102790 0x7fae201067e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae10009900 tx=0x7fae1002a410 comp rx=0 tx=0).stop 2024-09-12T02:15:46.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 -- 172.21.15.3:0/2698463129 shutdown_connections 2024-09-12T02:15:46.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 --2- 172.21.15.3:0/2698463129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 --2- 172.21.15.3:0/2698463129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20102790 0x7fae201067e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 --2- 172.21.15.3:0/2698463129 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20101de0 0x7fae201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 -- 172.21.15.3:0/2698463129 >> 172.21.15.3:0/2698463129 conn(0x7fae200fd8c0 msgr2=0x7fae200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 -- 172.21.15.3:0/2698463129 shutdown_connections 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 -- 172.21.15.3:0/2698463129 wait complete. 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.284+0000 7fae26b29640 1 Processor -- start 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 -- start start 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20102790 0x7fae20074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae20074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111f30 con 0x7fae20101de0 2024-09-12T02:15:46.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111db0 con 0x7fae20106f10 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae26b29640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fae201120b0 con 0x7fae20102790 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34318/0 (socket says 172.21.15.3:34318) 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 -- 172.21.15.3:0/1043313223 learned_addr learned my addr 172.21.15.3:0/1043313223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae1ffff640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20102790 0x7fae20074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae250c6640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae20074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 -- 172.21.15.3:0/1043313223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20102790 msgr2=0x7fae20074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20102790 0x7fae20074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 -- 172.21.15.3:0/1043313223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 msgr2=0x7fae20074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae20074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 -- 172.21.15.3:0/1043313223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae201a6980 con 0x7fae20101de0 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae250c6640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae20074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:15:46.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.285+0000 7fae248c5640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.286+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08017070 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.286+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae08009c30 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.286+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae201a6c70 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.286+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800f6b0 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.286+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a71b0 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.287+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae201021c0 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.288+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fae080109d0 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.289+0000 7fae1dffb640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.289+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fae080ce7e0 con 0x7fae20101de0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.289+0000 7fae1ffff640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:46.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.289+0000 7fae1ffff640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae201087b0 tx=0x7fae10005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:46.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.293+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae08009de0 con 0x7fae20101de0 2024-09-12T02:15:46.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.505+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2010a0f0 con 0x7fae20101de0 2024-09-12T02:15:46.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.505+0000 7fae1dffb640 1 -- 172.21.15.3:0/1043313223 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fae0809bbe0 con 0x7fae20101de0 2024-09-12T02:15:46.506 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:46.506 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"btime":"2024-09-12T02:08:39:464980+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T02:15:46.506 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2024-09-12T02:15:46.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 msgr2=0x7fae0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae201087b0 tx=0x7fae10005a20 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 msgr2=0x7fae200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 shutdown_connections 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fae0c0a2c00 0x7fae0c0a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae20106f10 0x7fae20074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae20102790 0x7fae20074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 --2- 172.21.15.3:0/1043313223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae20101de0 0x7fae200744f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 >> 172.21.15.3:0/1043313223 conn(0x7fae200fd8c0 msgr2=0x7fae200fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 shutdown_connections 2024-09-12T02:15:46.509 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:46.508+0000 7fae26b29640 1 -- 172.21.15.3:0/1043313223 wait complete. 2024-09-12T02:15:47.039 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:46 smithi092 systemd[1]: Starting Ceph mds.l for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:15:47.374 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 podman[65926]: 2024-09-12 02:15:47.026325309 +0000 UTC m=+0.015783032 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:15:47.374 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 podman[65926]: 2024-09-12 02:15:47.130415238 +0000 UTC m=+0.119872951 container create 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T02:15:47.374 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 podman[65926]: 2024-09-12 02:15:47.374814716 +0000 UTC m=+0.364272428 container init 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:15:47.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:47 smithi003 ceph-mon[31476]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:47.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:47 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1043313223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:47.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:47 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6832/3877701831,v1:172.21.15.92:6833/3877701831] up:boot 2024-09-12T02:15:47.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:47 smithi003 ceph-mon[31476]: fsmap 1 up:standby 2024-09-12T02:15:47.742 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:47 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:15:47.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:47 smithi026 ceph-mon[35496]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:47.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:47 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1043313223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:47.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:47 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6832/3877701831,v1:172.21.15.92:6833/3877701831] up:boot 2024-09-12T02:15:47.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:47 smithi026 ceph-mon[35496]: fsmap 1 up:standby 2024-09-12T02:15:47.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:47 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mon[36089]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1043313223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6832/3877701831,v1:172.21.15.92:6833/3877701831] up:boot 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mon[36089]: fsmap 1 up:standby 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 podman[65926]: 2024-09-12 02:15:47.377813558 +0000 UTC m=+0.367271273 container start 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-mds[65945]: -- 172.21.15.92:0/2706803112 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55896e7bd860 con 0x55896e79b400 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: starting mds.l at 2024-09-12T02:15:47.789 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 bash[65926]: 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e 2024-09-12T02:15:47.790 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:15:47 smithi092 systemd[1]: Started Ceph mds.l for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:48 smithi026 ceph-mon[35496]: Deploying daemon mds.k on smithi026 2024-09-12T02:15:49.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:49.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:49.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:49.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:48 smithi003 ceph-mon[31476]: Deploying daemon mds.k on smithi026 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:49.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:48 smithi092 ceph-mon[36089]: Deploying daemon mds.k on smithi026 2024-09-12T02:15:49.204 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:49.471 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:49.870 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:49 smithi003 ceph-mon[31476]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:49.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 -- 172.21.15.3:0/4267782992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8100a80 msgr2=0x7fcfe8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/4267782992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8100a80 0x7fcfe8100ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009cd0 tx=0x7fcfd802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 -- 172.21.15.3:0/4267782992 shutdown_connections 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/4267782992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8101420 0x7fcfe810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/4267782992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8100a80 0x7fcfe8100ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/4267782992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 0x7fcfe8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 -- 172.21.15.3:0/4267782992 >> 172.21.15.3:0/4267782992 conn(0x7fcfe80fc7a0 msgr2=0x7fcfe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 -- 172.21.15.3:0/4267782992 shutdown_connections 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.953+0000 7fcfeeb65640 1 -- 172.21.15.3:0/4267782992 wait complete. 2024-09-12T02:15:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 Processor -- start 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 -- start start 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8100a80 0x7fcfe8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 0x7fcfe8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8110e80 con 0x7fcfe8101420 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8110d00 con 0x7fcfe8106a80 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfeeb65640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8111000 con 0x7fcfe8100a80 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43074/0 (socket says 172.21.15.3:43074) 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 -- 172.21.15.3:0/3567784342 learned_addr learned my addr 172.21.15.3:0/3567784342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfec901640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8100a80 0x7fcfe8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:49.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfed102640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 0x7fcfe8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 -- 172.21.15.3:0/3567784342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8100a80 msgr2=0x7fcfe8197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8100a80 0x7fcfe8197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 -- 172.21.15.3:0/3567784342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 msgr2=0x7fcfe8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 0x7fcfe8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.954+0000 7fcfe7fff640 1 -- 172.21.15.3:0/3567784342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe81a6890 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfe7fff640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802ae00 tx=0x7fcfd8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004040 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81a6b20 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd80046c0 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8030e70 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.955+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81a70b0 con 0x7fcfe8101420 2024-09-12T02:15:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.956+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfe8100ee0 con 0x7fcfe8101420 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.958+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcfd80305a0 con 0x7fcfe8101420 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.959+0000 7fcfe5ffb640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 0x7fcfd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.959+0000 7fcfec901640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 0x7fcfd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.959+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcfd802e030 con 0x7fcfe8101420 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.959+0000 7fcfec901640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 0x7fcfd40a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000ab60 tx=0x7fcfd000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:49.962+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfd804b060 con 0x7fcfe8101420 2024-09-12T02:15:49.992 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:49 smithi026 ceph-mon[35496]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:49.992 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:49 smithi026 systemd[1]: Starting Ceph mds.k for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:15:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:49 smithi092 ceph-mon[36089]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.174+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfe806ad80 con 0x7fcfe8101420 2024-09-12T02:15:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.174+0000 7fcfe5ffb640 1 -- 172.21.15.3:0/3567784342 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fcfe806ad80 con 0x7fcfe8101420 2024-09-12T02:15:50.175 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:50.175 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"btime":"2024-09-12T02:15:47:406215+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:15:50.175 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-09-12T02:15:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 msgr2=0x7fcfd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 0x7fcfd40a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000ab60 tx=0x7fcfd000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 msgr2=0x7fcfe8197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802ae00 tx=0x7fcfd8003970 comp rx=0 tx=0).stop 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 shutdown_connections 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcfd40a2b80 0x7fcfd40a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfe8106a80 0x7fcfe8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcfe8101420 0x7fcfe8197560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 --2- 172.21.15.3:0/3567784342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe8100a80 0x7fcfe8197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:50.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 >> 172.21.15.3:0/3567784342 conn(0x7fcfe80fc7a0 msgr2=0x7fcfe80fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 shutdown_connections 2024-09-12T02:15:50.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:50.178+0000 7fcfeeb65640 1 -- 172.21.15.3:0/3567784342 wait complete. 2024-09-12T02:15:50.249 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:49 smithi026 podman[64705]: 2024-09-12 02:15:49.895222763 +0000 UTC m=+0.016036967 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:15:50.249 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 podman[64705]: 2024-09-12 02:15:50.021551117 +0000 UTC m=+0.142365304 container create 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:15:50.249 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 podman[64705]: 2024-09-12 02:15:50.249393872 +0000 UTC m=+0.370208069 container init 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T02:15:50.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 podman[64705]: 2024-09-12 02:15:50.252428909 +0000 UTC m=+0.373243097 container start 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:15:50.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mds[64724]: -- 172.21.15.26:0/258401042 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557547799860 con 0x557547777400 2024-09-12T02:15:50.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: starting mds.k at 2024-09-12T02:15:50.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 bash[64705]: 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 2024-09-12T02:15:50.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:15:50 smithi026 systemd[1]: Started Ceph mds.k for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3567784342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:50.943 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:50 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3567784342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:51.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:50 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:51.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3567784342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:51.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:50 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:51 smithi026 ceph-mon[35496]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:51 smithi026 ceph-mon[35496]: Deploying daemon mds.j on smithi003 2024-09-12T02:15:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:51 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6832/1804609961,v1:172.21.15.26:6833/1804609961] up:boot 2024-09-12T02:15:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:51 smithi026 ceph-mon[35496]: fsmap 2 up:standby 2024-09-12T02:15:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:51 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:15:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:51 smithi003 ceph-mon[31476]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:51 smithi003 ceph-mon[31476]: Deploying daemon mds.j on smithi003 2024-09-12T02:15:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:51 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6832/1804609961,v1:172.21.15.26:6833/1804609961] up:boot 2024-09-12T02:15:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:51 smithi003 ceph-mon[31476]: fsmap 2 up:standby 2024-09-12T02:15:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:51 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:15:52.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:51 smithi092 ceph-mon[36089]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:51 smithi092 ceph-mon[36089]: Deploying daemon mds.j on smithi003 2024-09-12T02:15:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:51 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6832/1804609961,v1:172.21.15.26:6833/1804609961] up:boot 2024-09-12T02:15:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:51 smithi092 ceph-mon[36089]: fsmap 2 up:standby 2024-09-12T02:15:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:51 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:15:52.778 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:52 smithi003 systemd[1]: Starting Ceph mds.j for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:15:52.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:53.216 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:52 smithi003 podman[71055]: 2024-09-12 02:15:52.826641573 +0000 UTC m=+0.016117591 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:15:53.216 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:52 smithi003 podman[71055]: 2024-09-12 02:15:52.929791335 +0000 UTC m=+0.119267380 container create 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T02:15:53.216 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 podman[71055]: 2024-09-12 02:15:53.216110221 +0000 UTC m=+0.405586224 container init 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:15:53.418 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:53.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 podman[71055]: 2024-09-12 02:15:53.219549057 +0000 UTC m=+0.409025068 container start 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:15:53.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 ceph-mds[71105]: -- 172.21.15.3:0/609246537 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56208c6d1860 con 0x56208c6af000 2024-09-12T02:15:53.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j[71089]: starting mds.j at 2024-09-12T02:15:53.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 bash[71055]: 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 2024-09-12T02:15:53.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:15:53 smithi003 systemd[1]: Started Ceph mds.j for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- 172.21.15.3:0/146246355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88105ad0 msgr2=0x7fba88105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/146246355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88105ad0 0x7fba88105eb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fba78009900 tx=0x7fba7802a410 comp rx=0 tx=0).stop 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- 172.21.15.3:0/146246355 shutdown_connections 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/146246355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88101ba0 0x7fba8810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/146246355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 0x7fba88101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/146246355 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88105ad0 0x7fba88105eb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- 172.21.15.3:0/146246355 >> 172.21.15.3:0/146246355 conn(0x7fba88077210 msgr2=0x7fba880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- 172.21.15.3:0/146246355 shutdown_connections 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- 172.21.15.3:0/146246355 wait complete. 2024-09-12T02:15:53.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 Processor -- start 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.878+0000 7fba8ecd2640 1 -- start start 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 0x7fba88198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88105ad0 0x7fba8819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba88110f70 con 0x7fba88101ba0 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba88110df0 con 0x7fba88105ad0 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ecd2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba881110f0 con 0x7fba88101200 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43100/0 (socket says 172.21.15.3:43100) 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 -- 172.21.15.3:0/1589232916 learned_addr learned my addr 172.21.15.3:0/1589232916 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:53.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8d26f640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88105ad0 0x7fba8819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba8ca6e640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 0x7fba88198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 -- 172.21.15.3:0/1589232916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 msgr2=0x7fba88198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 0x7fba88198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 -- 172.21.15.3:0/1589232916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88105ad0 msgr2=0x7fba8819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:53.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88105ad0 0x7fba8819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 -- 172.21.15.3:0/1589232916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8819d4c0 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.879+0000 7fba87fff640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b4a0 tx=0x7fba7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.880+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c014070 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.880+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c010af0 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.880+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8819d7b0 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.880+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c017d10 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.880+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba881a5090 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.881+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba881016e0 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.882+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fba7c017070 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.883+0000 7fba85ffb640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.883+0000 7fba8ca6e640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.883+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fba7c0cdf60 con 0x7fba88101ba0 2024-09-12T02:15:53.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.883+0000 7fba8ca6e640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 0x7fba700a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba78000c00 tx=0x7fba780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:53.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:53.887+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba7c09ac70 con 0x7fba88101ba0 2024-09-12T02:15:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.114+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba881a53b0 con 0x7fba88101ba0 2024-09-12T02:15:54.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.115+0000 7fba85ffb640 1 -- 172.21.15.3:0/1589232916 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fba7c09ef70 con 0x7fba88101ba0 2024-09-12T02:15:54.115 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:54.115 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"btime":"2024-09-12T02:15:53:687193+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:15:54.116 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 msgr2=0x7fba700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 0x7fba700a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fba78000c00 tx=0x7fba780058d0 comp rx=0 tx=0).stop 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 msgr2=0x7fba88198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b4a0 tx=0x7fba7c00b960 comp rx=0 tx=0).stop 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 shutdown_connections 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba700a2b80 0x7fba700a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:54.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba88105ad0 0x7fba8819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba88101ba0 0x7fba88198a50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 --2- 172.21.15.3:0/1589232916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba88101200 0x7fba88198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 >> 172.21.15.3:0/1589232916 conn(0x7fba88077210 msgr2=0x7fba881004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 shutdown_connections 2024-09-12T02:15:54.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:54.117+0000 7fba8ecd2640 1 -- 172.21.15.3:0/1589232916 wait complete. 2024-09-12T02:15:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: Deploying daemon mds.d on smithi003 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6834/3477371464,v1:172.21.15.3:6835/3477371464] up:boot 2024-09-12T02:15:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: fsmap 3 up:standby 2024-09-12T02:15:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:15:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1589232916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: Deploying daemon mds.d on smithi003 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6834/3477371464,v1:172.21.15.3:6835/3477371464] up:boot 2024-09-12T02:15:54.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: fsmap 3 up:standby 2024-09-12T02:15:54.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:15:54.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1589232916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: Deploying daemon mds.d on smithi003 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6834/3477371464,v1:172.21.15.3:6835/3477371464] up:boot 2024-09-12T02:15:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: fsmap 3 up:standby 2024-09-12T02:15:54.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:15:54.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1589232916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:56.363 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 systemd[1]: Starting Ceph mds.d for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:15:56.616 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:56 smithi003 ceph-mon[31476]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:56.616 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 podman[71473]: 2024-09-12 02:15:56.503604555 +0000 UTC m=+0.111572767 container create 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T02:15:56.616 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 podman[71473]: 2024-09-12 02:15:56.408187881 +0000 UTC m=+0.016156103 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:15:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:56 smithi026 ceph-mon[35496]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:56 smithi092 ceph-mon[36089]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:57.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 podman[71473]: 2024-09-12 02:15:56.731198789 +0000 UTC m=+0.339167009 container init 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 podman[71473]: 2024-09-12 02:15:56.73416317 +0000 UTC m=+0.342131383 container start 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 ceph-mds[71491]: -- 172.21.15.3:0/2565766874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b6e5e3860 con 0x563b6e5c0800 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d[71487]: starting mds.d at 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 bash[71473]: 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 2024-09-12T02:15:57.028 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:15:56 smithi003 systemd[1]: Started Ceph mds.d for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:15:57.272 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:15:57.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 -- 172.21.15.3:0/2815514049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764106f10 msgr2=0x7f376410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 --2- 172.21.15.3:0/2815514049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764106f10 0x7f376410ea90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f375c009ca0 tx=0x7f375c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 -- 172.21.15.3:0/2815514049 shutdown_connections 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 --2- 172.21.15.3:0/2815514049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764106f10 0x7f376410ea90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 --2- 172.21.15.3:0/2815514049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764102790 0x7f37641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 --2- 172.21.15.3:0/2815514049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764101de0 0x7f37641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.726+0000 7f376989e640 1 -- 172.21.15.3:0/2815514049 >> 172.21.15.3:0/2815514049 conn(0x7f37640fd8c0 msgr2=0x7f37640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- 172.21.15.3:0/2815514049 shutdown_connections 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- 172.21.15.3:0/2815514049 wait complete. 2024-09-12T02:15:57.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 Processor -- start 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- start start 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764102790 0x7f3764198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 0x7f376419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111f70 con 0x7f3764101de0 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3764111df0 con 0x7f3764106f10 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f376989e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f37641120f0 con 0x7f3764102790 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43138/0 (socket says 172.21.15.3:43138) 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.727+0000 7f3762ffd640 1 -- 172.21.15.3:0/3402209867 learned_addr learned my addr 172.21.15.3:0/3402209867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f37637fe640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 0x7f376419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:57.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f375bfff640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764102790 0x7f3764198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 -- 172.21.15.3:0/3402209867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764102790 msgr2=0x7f3764198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764102790 0x7f3764198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 -- 172.21.15.3:0/3402209867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 msgr2=0x7f376419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 0x7f376419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 -- 172.21.15.3:0/3402209867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419d3a0 con 0x7f3764101de0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f37637fe640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 0x7f376419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f3762ffd640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f375400d950 tx=0x7f375400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754017070 con 0x7f3764101de0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419d690 con 0x7f3764101de0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754009c30 con 0x7f3764101de0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375400f6b0 con 0x7f3764101de0 2024-09-12T02:15:57.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.728+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a4f70 con 0x7f3764101de0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.729+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37641021c0 con 0x7f3764101de0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.730+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f37540109d0 con 0x7f3764101de0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.731+0000 7f376889c640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.731+0000 7f375bfff640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.731+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f37540ce860 con 0x7f3764101de0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.732+0000 7f375bfff640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 0x7f37500a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37640fdce0 tx=0x7f374c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:15:57.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.735+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3754009de0 con 0x7f3764101de0 2024-09-12T02:15:57.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.948+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3764199750 con 0x7f3764101de0 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: Deploying daemon mds.e on smithi026 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6836/4187721968,v1:172.21.15.3:6837/4187721968] up:boot 2024-09-12T02:15:57.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: fsmap 4 up:standby 2024-09-12T02:15:57.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:15:57.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.948+0000 7f376889c640 1 -- 172.21.15.3:0/3402209867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f3764199750 con 0x7f3764101de0 2024-09-12T02:15:57.950 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:15:57.950 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"btime":"2024-09-12T02:15:57:363560+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 5 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.950+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 msgr2=0x7f37500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 0x7f37500a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37640fdce0 tx=0x7f374c005ed0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 msgr2=0x7f37641983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f375400d950 tx=0x7f375400de10 comp rx=0 tx=0).stop 2024-09-12T02:15:57.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 shutdown_connections 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3764106f10 0x7f376419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3764102790 0x7f3764198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 --2- 172.21.15.3:0/3402209867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3764101de0 0x7f37641983f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 >> 172.21.15.3:0/3402209867 conn(0x7f37640fd8c0 msgr2=0x7f37640feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 shutdown_connections 2024-09-12T02:15:57.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:15:57.951+0000 7f376989e640 1 -- 172.21.15.3:0/3402209867 wait complete. 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: Deploying daemon mds.e on smithi026 2024-09-12T02:15:58.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6836/4187721968,v1:172.21.15.3:6837/4187721968] up:boot 2024-09-12T02:15:58.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: fsmap 4 up:standby 2024-09-12T02:15:58.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: Deploying daemon mds.e on smithi026 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6836/4187721968,v1:172.21.15.3:6837/4187721968] up:boot 2024-09-12T02:15:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: fsmap 4 up:standby 2024-09-12T02:15:58.290 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:15:59.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:58 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3402209867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:59.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:58 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3402209867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:59.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:58 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3402209867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:15:59.764 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:15:59 smithi026 systemd[1]: Started Ceph mds.e for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:00.157 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:00.157 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.157 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.157 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.157 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:00.158 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:00.158 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:15:59 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:15:59 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:15:59 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:00.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:01.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:00 smithi003 ceph-mon[31476]: Deploying daemon mds.f on smithi092 2024-09-12T02:16:01.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:00 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6834/1866420590,v1:172.21.15.26:6835/1866420590] up:boot 2024-09-12T02:16:01.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:00 smithi003 ceph-mon[31476]: fsmap 5 up:standby 2024-09-12T02:16:01.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:00 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:16:01.160 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:00 smithi092 ceph-mon[36089]: Deploying daemon mds.f on smithi092 2024-09-12T02:16:01.160 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:00 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6834/1866420590,v1:172.21.15.26:6835/1866420590] up:boot 2024-09-12T02:16:01.160 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:00 smithi092 ceph-mon[36089]: fsmap 5 up:standby 2024-09-12T02:16:01.160 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:00 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:16:01.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:00 smithi026 ceph-mon[35496]: Deploying daemon mds.f on smithi092 2024-09-12T02:16:01.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:00 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6834/1866420590,v1:172.21.15.26:6835/1866420590] up:boot 2024-09-12T02:16:01.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:00 smithi026 ceph-mon[35496]: fsmap 5 up:standby 2024-09-12T02:16:01.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:00 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 -- 172.21.15.3:0/643894968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c106f10 msgr2=0x7f685c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 --2- 172.21.15.3:0/643894968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c106f10 0x7f685c10ea90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6854009ca0 tx=0x7f685402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 -- 172.21.15.3:0/643894968 shutdown_connections 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 --2- 172.21.15.3:0/643894968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c106f10 0x7f685c10ea90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 --2- 172.21.15.3:0/643894968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c102790 0x7f685c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 --2- 172.21.15.3:0/643894968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 -- 172.21.15.3:0/643894968 >> 172.21.15.3:0/643894968 conn(0x7f685c0fd8c0 msgr2=0x7f685c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 -- 172.21.15.3:0/643894968 shutdown_connections 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.443+0000 7f6861528640 1 -- 172.21.15.3:0/643894968 wait complete. 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 Processor -- start 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 -- start start 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c106f10 0x7f685c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f685c111f70 con 0x7f685c102790 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f685c111df0 con 0x7f685c101de0 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6861528640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1120f0 con 0x7f685c106f10 2024-09-12T02:16:01.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48584/0 (socket says 172.21.15.3:48584) 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 -- 172.21.15.3:0/2050745331 learned_addr learned my addr 172.21.15.3:0/2050745331 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f685b7fe640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c106f10 0x7f685c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f685affd640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 -- 172.21.15.3:0/2050745331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c106f10 msgr2=0x7f685c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c106f10 0x7f685c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 -- 172.21.15.3:0/2050745331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 msgr2=0x7f685c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 -- 172.21.15.3:0/2050745331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c19d2e0 con 0x7f685c102790 2024-09-12T02:16:01.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f685affd640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.444+0000 7f6853fff640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f684400d950 tx=0x7f684400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.445+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844017030 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.445+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6844004020 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.445+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c19d5d0 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.445+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440047a0 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.445+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c1a4eb0 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.446+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c1021c0 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.447+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f68440041d0 con 0x7f685c102790 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.448+0000 7f6858ff9640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 0x7f68480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.448+0000 7f685affd640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 0x7f68480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:01.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.448+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f68440ce5c0 con 0x7f685c102790 2024-09-12T02:16:01.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.448+0000 7f685affd640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 0x7f68480a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f684c00abe0 tx=0x7f684c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:01.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.452+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f684409b920 con 0x7f685c102790 2024-09-12T02:16:01.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.665+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f685c02d7c0 con 0x7f685c102790 2024-09-12T02:16:01.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.665+0000 7f6858ff9640 1 -- 172.21.15.3:0/2050745331 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7f684409b220 con 0x7f685c102790 2024-09-12T02:16:01.666 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:01.666 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"btime":"2024-09-12T02:15:59:878033+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:16:01.666 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-09-12T02:16:01.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 msgr2=0x7f68480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 0x7f68480a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f684c00abe0 tx=0x7f684c00a5c0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 msgr2=0x7f685c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f684400d950 tx=0x7f684400de10 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 shutdown_connections 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68480a2b80 0x7f68480a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f685c106f10 0x7f685c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f685c102790 0x7f685c198870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 --2- 172.21.15.3:0/2050745331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f685c101de0 0x7f685c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:01.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 >> 172.21.15.3:0/2050745331 conn(0x7f685c0fd8c0 msgr2=0x7f685c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:01.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 shutdown_connections 2024-09-12T02:16:01.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:01.668+0000 7f6861528640 1 -- 172.21.15.3:0/2050745331 wait complete. 2024-09-12T02:16:01.789 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:01 smithi092 systemd[1]: Starting Ceph mds.f for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:16:02.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:01 smithi003 ceph-mon[31476]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:02.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2050745331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:02.144 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:01 smithi092 ceph-mon[36089]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:02.144 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:01 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2050745331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:02.145 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:01 smithi092 podman[66229]: 2024-09-12 02:16:01.794558552 +0000 UTC m=+0.015850090 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:16:02.145 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:01 smithi092 podman[66229]: 2024-09-12 02:16:01.917337586 +0000 UTC m=+0.138629114 container create fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:16:02.145 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 podman[66229]: 2024-09-12 02:16:02.145161883 +0000 UTC m=+0.366453414 container init fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T02:16:02.170 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:01 smithi026 ceph-mon[35496]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:02.170 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:01 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2050745331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:02.539 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 podman[66229]: 2024-09-12 02:16:02.148407892 +0000 UTC m=+0.369699421 container start fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T02:16:02.539 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 ceph-mds[66249]: -- 172.21.15.92:0/425150382 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d564237860 con 0x55d564215400 2024-09-12T02:16:02.539 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: starting mds.f at 2024-09-12T02:16:02.539 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 bash[66229]: fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 2024-09-12T02:16:02.539 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:16:02 smithi092 systemd[1]: Started Ceph mds.f for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: Deploying daemon mds.c on smithi092 2024-09-12T02:16:03.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:boot 2024-09-12T02:16:03.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: fsmap 6 up:standby 2024-09-12T02:16:03.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:03 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: Deploying daemon mds.c on smithi092 2024-09-12T02:16:03.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:boot 2024-09-12T02:16:03.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: fsmap 6 up:standby 2024-09-12T02:16:03.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:03 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: Deploying daemon mds.c on smithi092 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:boot 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: fsmap 6 up:standby 2024-09-12T02:16:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:03 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:16:04.424 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:04.610 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:04 smithi092 ceph-mon[36089]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:04.611 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 systemd[1]: Starting Ceph mds.c for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:16:04.691 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:04 smithi026 ceph-mon[35496]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:04.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:04 smithi003 ceph-mon[31476]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:04.886 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 podman[66529]: 2024-09-12 02:16:04.513416368 +0000 UTC m=+0.015608204 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:16:04.886 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 podman[66529]: 2024-09-12 02:16:04.631330495 +0000 UTC m=+0.133522374 container create 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1348108298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 msgr2=0x7fb8d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1348108298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4100ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009930 tx=0x7fb8bc02a420 comp rx=0 tx=0).stop 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1348108298 shutdown_connections 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1348108298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4101420 0x7fb8d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1348108298 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4100ee0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1348108298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4106a80 0x7fb8d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1348108298 >> 172.21.15.3:0/1348108298 conn(0x7fb8d40fc7a0 msgr2=0x7fb8d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1348108298 shutdown_connections 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1348108298 wait complete. 2024-09-12T02:16:05.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 Processor -- start 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.186+0000 7fb8da8e4640 1 -- start start 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 0x7fb8d4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4106a80 0x7fb8d41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4110e80 con 0x7fb8d4100a80 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4110d00 con 0x7fb8d4106a80 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4111000 con 0x7fb8d4101420 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48604/0 (socket says 172.21.15.3:48604) 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 -- 172.21.15.3:0/1282631643 learned_addr learned my addr 172.21.15.3:0/1282631643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d8e81640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4106a80 0x7fb8d41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:05.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8cbfff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 0x7fb8d4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 -- 172.21.15.3:0/1282631643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 msgr2=0x7fb8d4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 0x7fb8d4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 -- 172.21.15.3:0/1282631643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4106a80 msgr2=0x7fb8d41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4106a80 0x7fb8d41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 -- 172.21.15.3:0/1282631643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d41a48d0 con 0x7fb8d4100a80 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8cbfff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 0x7fb8d4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d3fff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400d950 tx=0x7fb8c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4017070 con 0x7fb8d4100a80 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d41a4aa0 con 0x7fb8d4100a80 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.187+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4009c30 con 0x7fb8d4100a80 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.188+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400f6b0 con 0x7fb8d4100a80 2024-09-12T02:16:05.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.188+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d41a4fe0 con 0x7fb8d4100a80 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.189+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d4100ee0 con 0x7fb8d4100a80 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.190+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb8c40109d0 con 0x7fb8d4100a80 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.190+0000 7fb8d1ffb640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 0x7fb8c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.190+0000 7fb8cbfff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 0x7fb8c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.190+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb8c40ce770 con 0x7fb8d4100a80 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.191+0000 7fb8cbfff640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 0x7fb8c00a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc000c00 tx=0x7fb8bc032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:05.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.195+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8c4009de0 con 0x7fb8d4100a80 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 podman[66529]: 2024-09-12 02:16:04.886454781 +0000 UTC m=+0.388646617 container init 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 podman[66529]: 2024-09-12 02:16:04.889364695 +0000 UTC m=+0.391556532 container start 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 ceph-mds[66548]: -- 172.21.15.92:0/3419250468 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590721b1860 con 0x55907218f000 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:04 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: starting mds.c at 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:05 smithi092 bash[66529]: 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 2024-09-12T02:16:05.289 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:16:05 smithi092 systemd[1]: Started Ceph mds.c for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:05.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.408+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8d4108ff0 con 0x7fb8d4100a80 2024-09-12T02:16:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.409+0000 7fb8d1ffb640 1 -- 172.21.15.3:0/1282631643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6376 (secure 0 0 0) 0x7fb8c409bb70 con 0x7fb8d4100a80 2024-09-12T02:16:05.409 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:05.409 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"btime":"2024-09-12T02:16:05:318140+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T02:16:05.409 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 msgr2=0x7fb8c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 0x7fb8c00a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc000c00 tx=0x7fb8bc032c70 comp rx=0 tx=0).stop 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 msgr2=0x7fb8d4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400d950 tx=0x7fb8c400de10 comp rx=0 tx=0).stop 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 shutdown_connections 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8c00a2b80 0x7fb8c00a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d4106a80 0x7fb8d41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8d4101420 0x7fb8d4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 --2- 172.21.15.3:0/1282631643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8d4100a80 0x7fb8d4199310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 >> 172.21.15.3:0/1282631643 conn(0x7fb8d40fc7a0 msgr2=0x7fb8d40fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 shutdown_connections 2024-09-12T02:16:05.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:05.411+0000 7fb8da8e4640 1 -- 172.21.15.3:0/1282631643 wait complete. 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: Deploying daemon mds.h on smithi026 2024-09-12T02:16:06.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:boot 2024-09-12T02:16:06.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: fsmap 7 up:standby 2024-09-12T02:16:06.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:16:06.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:06 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1282631643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: Deploying daemon mds.h on smithi026 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:boot 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: fsmap 7 up:standby 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:16:06.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:06 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1282631643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: Deploying daemon mds.h on smithi026 2024-09-12T02:16:06.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:boot 2024-09-12T02:16:06.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: fsmap 7 up:standby 2024-09-12T02:16:06.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:16:06.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:06 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1282631643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:07.495 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 systemd[1]: Starting Ceph mds.h for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:16:07.765 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 podman[65319]: 2024-09-12 02:16:07.398062373 +0000 UTC m=+0.016094620 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:16:07.765 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 podman[65319]: 2024-09-12 02:16:07.508209924 +0000 UTC m=+0.126242159 container create 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:16:08.049 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 podman[65319]: 2024-09-12 02:16:07.769307193 +0000 UTC m=+0.387339429 container init 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 podman[65319]: 2024-09-12 02:16:07.772016249 +0000 UTC m=+0.390048490 container start 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 ceph-mds[65338]: -- 172.21.15.26:0/2270459652 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e817261860 con 0x55e81723f400 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: starting mds.h at 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 bash[65319]: 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a 2024-09-12T02:16:08.062 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:16:07 smithi026 systemd[1]: Started Ceph mds.h for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:08.310 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:08.381 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:08.382 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:08 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:08.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:08.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.431 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:08.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:08.432 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:08 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:08.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:08 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 -- 172.21.15.3:0/2541773883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 msgr2=0x7f0fb41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 --2- 172.21.15.3:0/2541773883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb41021c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009900 tx=0x7f0fa402a410 comp rx=0 tx=0).stop 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 -- 172.21.15.3:0/2541773883 shutdown_connections 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 --2- 172.21.15.3:0/2541773883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4106f10 0x7f0fb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 --2- 172.21.15.3:0/2541773883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4102790 0x7f0fb41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 --2- 172.21.15.3:0/2541773883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb41021c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:08.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 -- 172.21.15.3:0/2541773883 >> 172.21.15.3:0/2541773883 conn(0x7f0fb40fd8c0 msgr2=0x7f0fb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 -- 172.21.15.3:0/2541773883 shutdown_connections 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.845+0000 7f0fb917f640 1 -- 172.21.15.3:0/2541773883 wait complete. 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 Processor -- start 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 -- start start 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4102790 0x7f0fb4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4106f10 0x7f0fb419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4111f30 con 0x7f0fb4101de0 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb4111db0 con 0x7f0fb4102790 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.846+0000 7f0fb917f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb41120b0 con 0x7f0fb4106f10 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40574/0 (socket says 172.21.15.3:40574) 2024-09-12T02:16:08.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 -- 172.21.15.3:0/644084143 learned_addr learned my addr 172.21.15.3:0/644084143 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb259c640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4102790 0x7f0fb4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb359e640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4106f10 0x7f0fb419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 -- 172.21.15.3:0/644084143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4106f10 msgr2=0x7f0fb419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4106f10 0x7f0fb419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 -- 172.21.15.3:0/644084143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4102790 msgr2=0x7f0fb4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4102790 0x7f0fb4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 -- 172.21.15.3:0/644084143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb419d4c0 con 0x7f0fb4101de0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb2d9d640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4006010 tx=0x7f0fa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa403f030 con 0x7f0fb4101de0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb419d750 con 0x7f0fb4101de0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa4003ce0 con 0x7f0fb4101de0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4035070 con 0x7f0fb4101de0 2024-09-12T02:16:08.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.847+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb41a5090 con 0x7f0fb4101de0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.848+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb41032c0 con 0x7f0fb4101de0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.850+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0fa4042030 con 0x7f0fb4101de0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.851+0000 7f0f9bfff640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 0x7f0f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.851+0000 7f0fb259c640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 0x7f0f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.851+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f0fa40ef450 con 0x7f0fb4101de0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.852+0000 7f0fb259c640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 0x7f0f9c0a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0fb41087b0 tx=0x7f0fa800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:08.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:08.854+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fa40bc2c0 con 0x7f0fb4101de0 2024-09-12T02:16:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.065+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fb41021c0 con 0x7f0fb4101de0 2024-09-12T02:16:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.066+0000 7f0f9bfff640 1 -- 172.21.15.3:0/644084143 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7201 (secure 0 0 0) 0x7f0fa40c05c0 con 0x7f0fb4101de0 2024-09-12T02:16:09.066 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-09-12T02:16:09.067 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:09.067 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"btime":"2024-09-12T02:16:08:053927+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T02:16:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 msgr2=0x7f0f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:09.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 0x7f0f9c0a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0fb41087b0 tx=0x7f0fa800b390 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 msgr2=0x7f0fb4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4006010 tx=0x7f0fa40042f0 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 shutdown_connections 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0f9c0a2b80 0x7f0f9c0a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0fb4106f10 0x7f0fb419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fb4102790 0x7f0fb4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 --2- 172.21.15.3:0/644084143 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0fb4101de0 0x7f0fb4198510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.068+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 >> 172.21.15.3:0/644084143 conn(0x7f0fb40fd8c0 msgr2=0x7f0fb40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.069+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 shutdown_connections 2024-09-12T02:16:09.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:09.069+0000 7f0fb917f640 1 -- 172.21.15.3:0/644084143 wait complete. 2024-09-12T02:16:09.407 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:09 smithi092 ceph-mon[36089]: Deploying daemon mds.i on smithi092 2024-09-12T02:16:09.407 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:09 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6836/2891911183,v1:172.21.15.26:6837/2891911183] up:boot 2024-09-12T02:16:09.407 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:09 smithi092 ceph-mon[36089]: fsmap 8 up:standby 2024-09-12T02:16:09.407 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:09 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:16:09.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:09 smithi003 ceph-mon[31476]: Deploying daemon mds.i on smithi092 2024-09-12T02:16:09.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:09 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6836/2891911183,v1:172.21.15.26:6837/2891911183] up:boot 2024-09-12T02:16:09.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:09 smithi003 ceph-mon[31476]: fsmap 8 up:standby 2024-09-12T02:16:09.479 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:09 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:16:09.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:09 smithi026 ceph-mon[35496]: Deploying daemon mds.i on smithi092 2024-09-12T02:16:09.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:09 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6836/2891911183,v1:172.21.15.26:6837/2891911183] up:boot 2024-09-12T02:16:09.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:09 smithi026 ceph-mon[35496]: fsmap 8 up:standby 2024-09-12T02:16:09.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:09 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:16:10.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:10 smithi092 ceph-mon[36089]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:10.241 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:10 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/644084143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:10 smithi026 ceph-mon[35496]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:10 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/644084143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:10.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:10 smithi003 ceph-mon[31476]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:10.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:10 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/644084143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:10.788 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:10 smithi092 systemd[1]: Started Ceph mds.i for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:11.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:11.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:11.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:11.757 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:11.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:11.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:11.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:11.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:11.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: Deploying daemon mds.g on smithi003 2024-09-12T02:16:11.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:boot 2024-09-12T02:16:11.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: fsmap 9 up:standby 2024-09-12T02:16:11.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:11 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: Deploying daemon mds.g on smithi003 2024-09-12T02:16:12.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:boot 2024-09-12T02:16:12.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: fsmap 9 up:standby 2024-09-12T02:16:12.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:11 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:12.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: Deploying daemon mds.g on smithi003 2024-09-12T02:16:12.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:boot 2024-09-12T02:16:12.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: fsmap 9 up:standby 2024-09-12T02:16:12.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:11 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:16:12.220 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.823+0000 7f67caf09640 1 -- 172.21.15.3:0/665438878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4100350 msgr2=0x7f67c4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.823+0000 7f67caf09640 1 --2- 172.21.15.3:0/665438878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4100350 0x7f67c4100730 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009930 tx=0x7f67b802a420 comp rx=0 tx=0).stop 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.823+0000 7f67caf09640 1 -- 172.21.15.3:0/665438878 shutdown_connections 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.823+0000 7f67caf09640 1 --2- 172.21.15.3:0/665438878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4106f50 0x7f67c410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 --2- 172.21.15.3:0/665438878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:12.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 --2- 172.21.15.3:0/665438878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4100350 0x7f67c4100730 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- 172.21.15.3:0/665438878 >> 172.21.15.3:0/665438878 conn(0x7f67c4077210 msgr2=0x7f67c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- 172.21.15.3:0/665438878 shutdown_connections 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- 172.21.15.3:0/665438878 wait complete. 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 Processor -- start 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- start start 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4100350 0x7f67c41012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112000 con 0x7f67c4106f50 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4111e80 con 0x7f67c4100d00 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.824+0000 7f67caf09640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4112180 con 0x7f67c4100350 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:12.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40604/0 (socket says 172.21.15.3:40604) 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 -- 172.21.15.3:0/3297716743 learned_addr learned my addr 172.21.15.3:0/3297716743 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c3fff640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c8ca5640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4100350 0x7f67c41012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 -- 172.21.15.3:0/3297716743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4100350 msgr2=0x7f67c41012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4100350 0x7f67c41012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 -- 172.21.15.3:0/3297716743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 msgr2=0x7f67c4101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 -- 172.21.15.3:0/3297716743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41a69e0 con 0x7f67c4106f50 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c3fff640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c94a6640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b970 tx=0x7f67bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc019070 con 0x7f67c4106f50 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67bc00cb40 con 0x7f67c4106f50 2024-09-12T02:16:12.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41a6cd0 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc007650 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.825+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c41a71c0 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.827+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c4100730 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.828+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f67bc01c070 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.828+0000 7f67c1ffb640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 0x7f67b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.828+0000 7f67c8ca5640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 0x7f67b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.829+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f67bc0ce230 con 0x7f67c4106f50 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.829+0000 7f67c8ca5640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 0x7f67b40a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000c00 tx=0x7f67b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:12.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:12.833+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67bc09b630 con 0x7f67c4106f50 2024-09-12T02:16:13.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.053+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c410a140 con 0x7f67c4106f50 2024-09-12T02:16:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.054+0000 7f67c1ffb640 1 -- 172.21.15.3:0/3297716743 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8028 (secure 0 0 0) 0x7f67bc09af30 con 0x7f67c4106f50 2024-09-12T02:16:13.054 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:13.054 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":10,"btime":"2024-09-12T02:16:11:069207+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:16:13.054 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 10 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 msgr2=0x7f67b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 0x7f67b40a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67b8000c00 tx=0x7f67b8005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 msgr2=0x7f67c4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b970 tx=0x7f67bc00be30 comp rx=0 tx=0).stop 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 shutdown_connections 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f67b40a2b80 0x7f67b40a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:13.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f67c4106f50 0x7f67c4101d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67c4100d00 0x7f67c4101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 --2- 172.21.15.3:0/3297716743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67c4100350 0x7f67c41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 >> 172.21.15.3:0/3297716743 conn(0x7f67c4077210 msgr2=0x7f67c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 shutdown_connections 2024-09-12T02:16:13.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:13.056+0000 7f67caf09640 1 -- 172.21.15.3:0/3297716743 wait complete. 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3297716743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:13.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:13 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3297716743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:13 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3297716743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:14.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:14.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:13 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:14 smithi026 ceph-mon[35496]: Deploying daemon mds.a on smithi003 2024-09-12T02:16:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:14 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6838/775355044,v1:172.21.15.3:6839/775355044] up:boot 2024-09-12T02:16:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:14 smithi026 ceph-mon[35496]: fsmap 10 up:standby 2024-09-12T02:16:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:14 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:16:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:14 smithi003 ceph-mon[31476]: Deploying daemon mds.a on smithi003 2024-09-12T02:16:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:14 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6838/775355044,v1:172.21.15.3:6839/775355044] up:boot 2024-09-12T02:16:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:14 smithi003 ceph-mon[31476]: fsmap 10 up:standby 2024-09-12T02:16:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:14 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:16:15.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:14 smithi092 ceph-mon[36089]: Deploying daemon mds.a on smithi003 2024-09-12T02:16:15.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:14 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6838/775355044,v1:172.21.15.3:6839/775355044] up:boot 2024-09-12T02:16:15.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:14 smithi092 ceph-mon[36089]: fsmap 10 up:standby 2024-09-12T02:16:15.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:14 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:16:16.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:15 smithi026 ceph-mon[35496]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:16.022 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:15 smithi003 ceph-mon[31476]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:16.023 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:15 smithi003 systemd[1]: Starting Ceph mds.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:16:16.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:15 smithi092 ceph-mon[36089]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:16.042 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:16.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 podman[72657]: 2024-09-12 02:16:15.925751472 +0000 UTC m=+0.016243321 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:16:16.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 podman[72657]: 2024-09-12 02:16:16.060490743 +0000 UTC m=+0.150982585 container create 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T02:16:16.490 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:16.612 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 podman[72657]: 2024-09-12 02:16:16.280214152 +0000 UTC m=+0.370705992 container init 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T02:16:16.612 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 podman[72657]: 2024-09-12 02:16:16.283414516 +0000 UTC m=+0.373906362 container start 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T02:16:16.612 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 ceph-mds[72708]: -- 172.21.15.3:0/570632676 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557024bf3860 con 0x557024bd1400 2024-09-12T02:16:16.613 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: starting mds.a at 2024-09-12T02:16:16.613 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 bash[72657]: 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 2024-09-12T02:16:16.613 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:16:16 smithi003 systemd[1]: Started Ceph mds.a for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 -- 172.21.15.3:0/270511132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd4101df0 msgr2=0x7febd41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:17.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 --2- 172.21.15.3:0/270511132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd4101df0 0x7febd41021d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7febc4009900 tx=0x7febc402a410 comp rx=0 tx=0).stop 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 -- 172.21.15.3:0/270511132 shutdown_connections 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 --2- 172.21.15.3:0/270511132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 0x7febd410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 --2- 172.21.15.3:0/270511132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd41027a0 0x7febd41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 --2- 172.21.15.3:0/270511132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd4101df0 0x7febd41021d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 -- 172.21.15.3:0/270511132 >> 172.21.15.3:0/270511132 conn(0x7febd40fd8b0 msgr2=0x7febd40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 -- 172.21.15.3:0/270511132 shutdown_connections 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.977+0000 7febdb25e640 1 -- 172.21.15.3:0/270511132 wait complete. 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 Processor -- start 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 -- start start 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd4101df0 0x7febd40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 0x7febd4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febd4111f80 con 0x7febd41027a0 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7febd4111e00 con 0x7febd4106f20 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febdb25e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7febd4112100 con 0x7febd4101df0 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febd3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febd3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40640/0 (socket says 172.21.15.3:40640) 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febd3fff640 1 -- 172.21.15.3:0/1017324540 learned_addr learned my addr 172.21.15.3:0/1017324540 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.978+0000 7febd97fb640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 0x7febd4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd8ffa640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd4101df0 0x7febd40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 -- 172.21.15.3:0/1017324540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd4101df0 msgr2=0x7febd40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd4101df0 0x7febd40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 -- 172.21.15.3:0/1017324540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 msgr2=0x7febd4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 0x7febd4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 -- 172.21.15.3:0/1017324540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd41a6930 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd3fff640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d950 tx=0x7febbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc017070 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febbc009c30 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc00f6b0 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd41a6c20 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.979+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41a7110 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.980+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd41021d0 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.981+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7febbc0109d0 con 0x7febd41027a0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.982+0000 7febd1ffb640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 0x7febc00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.982+0000 7febd8ffa640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 0x7febc00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:17.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.982+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7febbc0ce730 con 0x7febd41027a0 2024-09-12T02:16:17.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.982+0000 7febd8ffa640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 0x7febc00a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7febc4005ab0 tx=0x7febc4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:17.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:16.986+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febbc009de0 con 0x7febd41027a0 2024-09-12T02:16:17.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.199+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd410a100 con 0x7febd41027a0 2024-09-12T02:16:17.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.200+0000 7febd1ffb640 1 -- 172.21.15.3:0/1017324540 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7febbc09bb30 con 0x7febd41027a0 2024-09-12T02:16:17.200 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:17.200 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"btime":"2024-09-12T02:16:16:663732+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:16:17.201 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2024-09-12T02:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 msgr2=0x7febc00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 0x7febc00a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7febc4005ab0 tx=0x7febc4005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 msgr2=0x7febd4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:17.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7febbc00d950 tx=0x7febbc00de10 comp rx=0 tx=0).stop 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 shutdown_connections 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7febc00a2c00 0x7febc00a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febd4106f20 0x7febd4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febd41027a0 0x7febd4074a30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 --2- 172.21.15.3:0/1017324540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd4101df0 0x7febd40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 >> 172.21.15.3:0/1017324540 conn(0x7febd40fd8b0 msgr2=0x7febd40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 shutdown_connections 2024-09-12T02:16:17.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:17.202+0000 7febdb25e640 1 -- 172.21.15.3:0/1017324540 wait complete. 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:17.501 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:17.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: Deploying daemon mds.b on smithi026 2024-09-12T02:16:17.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:boot 2024-09-12T02:16:17.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: fsmap 11 up:standby 2024-09-12T02:16:17.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:16:17.502 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:17 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1017324540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:17.504 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:17.504 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: Deploying daemon mds.b on smithi026 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:boot 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: fsmap 11 up:standby 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:16:17.505 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:17 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1017324540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:16:17.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: Deploying daemon mds.b on smithi026 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:boot 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: fsmap 11 up:standby 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:16:17.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:17 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1017324540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:18.765 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:18 smithi026 systemd[1]: Starting Ceph mds.b for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:16:19.150 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:18 smithi026 podman[65632]: 2024-09-12 02:16:18.789074962 +0000 UTC m=+0.016356748 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:16:19.150 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:18 smithi026 podman[65632]: 2024-09-12 02:16:18.90607868 +0000 UTC m=+0.133360450 container create f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True) 2024-09-12T02:16:19.514 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 podman[65632]: 2024-09-12 02:16:19.150547937 +0000 UTC m=+0.377829718 container init f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T02:16:19.515 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 podman[65632]: 2024-09-12 02:16:19.153557334 +0000 UTC m=+0.380839105 container start f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T02:16:19.515 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 ceph-mds[65651]: -- 172.21.15.26:0/3767506050 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eea4955860 con 0x55eea4932800 2024-09-12T02:16:19.515 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: starting mds.b at 2024-09-12T02:16:19.515 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 bash[65632]: f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 2024-09-12T02:16:19.515 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:16:19 smithi026 systemd[1]: Started Ceph mds.b for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:16:19.826 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:20.311 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:20.311 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.312 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.312 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.312 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.312 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.313 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:20.313 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:boot 2024-09-12T02:16:20.313 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: fsmap 12 up:standby 2024-09-12T02:16:20.313 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:20 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:16:20.737 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:boot 2024-09-12T02:16:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: fsmap 12 up:standby 2024-09-12T02:16:20.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:20 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:16:20.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:20.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:boot 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: fsmap 12 up:standby 2024-09-12T02:16:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:20 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:16:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 -- 172.21.15.3:0/2967434544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c100e20 msgr2=0x7f322c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 --2- 172.21.15.3:0/2967434544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c100e20 0x7f322c101200 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02a480 comp rx=0 tx=0).stop 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 -- 172.21.15.3:0/2967434544 shutdown_connections 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 --2- 172.21.15.3:0/2967434544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c102ef0 0x7f322c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 --2- 172.21.15.3:0/2967434544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102570 0x7f322c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 --2- 172.21.15.3:0/2967434544 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c100e20 0x7f322c101200 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 -- 172.21.15.3:0/2967434544 >> 172.21.15.3:0/2967434544 conn(0x7f322c0fc7d0 msgr2=0x7f322c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 -- 172.21.15.3:0/2967434544 shutdown_connections 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 -- 172.21.15.3:0/2967434544 wait complete. 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.238+0000 7f3232419640 1 Processor -- start 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 -- start start 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 0x7f322c197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102ef0 0x7f322c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f322c110f80 con 0x7f322c102570 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f322c110e00 con 0x7f322c100e20 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f3232419640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f322c111100 con 0x7f322c102ef0 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50168/0 (socket says 172.21.15.3:50168) 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 -- 172.21.15.3:0/2433779724 learned_addr learned my addr 172.21.15.3:0/2433779724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f32309b6640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102ef0 0x7f322c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322bfff640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 0x7f322c197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 -- 172.21.15.3:0/2433779724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102ef0 msgr2=0x7f322c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102ef0 0x7f322c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 -- 172.21.15.3:0/2433779724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 msgr2=0x7f322c197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 0x7f322c197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322b7fe640 1 -- 172.21.15.3:0/2433779724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c1a69c0 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.239+0000 7f322bfff640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 0x7f322c197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f322b7fe640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9f0 tx=0x7f322000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000cd80 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220010910 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c1a6cb0 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220018990 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.240+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a71a0 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.241+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322c101200 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.242+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3220010ac0 con 0x7f322c102570 2024-09-12T02:16:21.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.243+0000 7f32297fa640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 0x7f32140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.243+0000 7f322bfff640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 0x7f32140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.243+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f32200cf330 con 0x7f322c102570 2024-09-12T02:16:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.244+0000 7f322bfff640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 0x7f32140a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f321c005ab0 tx=0x7f321c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:21.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.246+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f322009c730 con 0x7f322c102570 2024-09-12T02:16:21.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.493+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322c06a560 con 0x7f322c102570 2024-09-12T02:16:21.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.493+0000 7f32297fa640 1 -- 172.21.15.3:0/2433779724 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7f322c06a560 con 0x7f322c102570 2024-09-12T02:16:21.494 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:21.494 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"btime":"2024-09-12T02:16:19:443172+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:16:21.494 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2024-09-12T02:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 msgr2=0x7f32140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 0x7f32140a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f321c005ab0 tx=0x7f321c005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 msgr2=0x7f322c197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:21.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9f0 tx=0x7f322000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 shutdown_connections 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32140a2b80 0x7f32140a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f322c102ef0 0x7f322c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f322c102570 0x7f322c197690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 --2- 172.21.15.3:0/2433779724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f322c100e20 0x7f322c197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 >> 172.21.15.3:0/2433779724 conn(0x7f322c0fc7d0 msgr2=0x7f322c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 shutdown_connections 2024-09-12T02:16:21.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:21.496+0000 7f3232419640 1 -- 172.21.15.3:0/2433779724 wait complete. 2024-09-12T02:16:22.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:22.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2433779724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.164 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:21 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:22.191 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T02:16:22.222 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2433779724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.223 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:21 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:22.232 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2433779724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:22.233 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:21 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:16:23.040 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 -- 172.21.15.3:0/2773367277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978100100 msgr2=0x7f9978100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 --2- 172.21.15.3:0/2773367277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978100100 0x7f9978100560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9960009900 tx=0x7f996002a410 comp rx=0 tx=0).stop 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 -- 172.21.15.3:0/2773367277 shutdown_connections 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 --2- 172.21.15.3:0/2773367277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 0x7f997810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 --2- 172.21.15.3:0/2773367277 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978100100 0x7f9978100560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 --2- 172.21.15.3:0/2773367277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978106b80 0x7f9978106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 -- 172.21.15.3:0/2773367277 >> 172.21.15.3:0/2773367277 conn(0x7f9978077230 msgr2=0x7f99780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 -- 172.21.15.3:0/2773367277 shutdown_connections 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 -- 172.21.15.3:0/2773367277 wait complete. 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.534+0000 7f997d968640 1 Processor -- start 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 -- start start 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100100 0x7f99781012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 0x7f9978101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f99781120a0 con 0x7f9978106b80 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9978111f20 con 0x7f9978100aa0 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f997d968640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9978112220 con 0x7f9978100100 2024-09-12T02:16:23.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50180/0 (socket says 172.21.15.3:50180) 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 -- 172.21.15.3:0/696369793 learned_addr learned my addr 172.21.15.3:0/696369793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f9976ffd640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100100 0x7f99781012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99767fc640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 0x7f9978101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 -- 172.21.15.3:0/696369793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100100 msgr2=0x7f99781012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100100 0x7f99781012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 -- 172.21.15.3:0/696369793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 msgr2=0x7f9978101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 0x7f9978101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.535+0000 7f99777fe640 1 -- 172.21.15.3:0/696369793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99781a69c0 con 0x7f9978106b80 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f99777fe640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f997000c970 tx=0x7f997000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970004230 con 0x7f9978106b80 2024-09-12T02:16:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99781a6cb0 con 0x7f9978106b80 2024-09-12T02:16:23.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99700043d0 con 0x7f9978106b80 2024-09-12T02:16:23.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997000f450 con 0x7f9978106b80 2024-09-12T02:16:23.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.536+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99781a71c0 con 0x7f9978106b80 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.537+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9978100560 con 0x7f9978106b80 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.538+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f99700079b0 con 0x7f9978106b80 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.539+0000 7f997c966640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 0x7f99640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.539+0000 7f9976ffd640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 0x7f99640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.539+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f99700cec90 con 0x7f9978106b80 2024-09-12T02:16:23.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.540+0000 7f9976ffd640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 0x7f99640a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9968005c40 tx=0x7f996800a570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:23.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.542+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9970014070 con 0x7f9978106b80 2024-09-12T02:16:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.741+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f997810a050 con 0x7f9978106b80 2024-09-12T02:16:23.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.741+0000 7f997c966640 1 -- 172.21.15.3:0/696369793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f997009c090 con 0x7f9978106b80 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:16:23.742 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.743+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 msgr2=0x7f99640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.743+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 0x7f99640a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9968005c40 tx=0x7f996800a570 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 msgr2=0x7f9978101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f997000c970 tx=0x7f997000ce30 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 shutdown_connections 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f99640a2b80 0x7f99640a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9978106b80 0x7f9978101d60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9978100aa0 0x7f9978101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 --2- 172.21.15.3:0/696369793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9978100100 0x7f99781012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:23.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 >> 172.21.15.3:0/696369793 conn(0x7f9978077230 msgr2=0x7f9978104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 shutdown_connections 2024-09-12T02:16:23.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:23.744+0000 7f997d968640 1 -- 172.21.15.3:0/696369793 wait complete. 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.365 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:24 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/696369793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:16:24.392 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T02:16:24.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:24 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/696369793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:24 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/696369793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:16:24.652 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 -- 172.21.15.3:0/4164046097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34102d40 msgr2=0x7ffa34103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 --2- 172.21.15.3:0/4164046097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34102d40 0x7ffa34103120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009900 tx=0x7ffa2402a480 comp rx=0 tx=0).stop 2024-09-12T02:16:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 -- 172.21.15.3:0/4164046097 shutdown_connections 2024-09-12T02:16:25.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 --2- 172.21.15.3:0/4164046097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 --2- 172.21.15.3:0/4164046097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34103660 0x7ffa3410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 --2- 172.21.15.3:0/4164046097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34102d40 0x7ffa34103120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 -- 172.21.15.3:0/4164046097 >> 172.21.15.3:0/4164046097 conn(0x7ffa340fc7a0 msgr2=0x7ffa340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 -- 172.21.15.3:0/4164046097 shutdown_connections 2024-09-12T02:16:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.105+0000 7ffa388ce640 1 -- 172.21.15.3:0/4164046097 wait complete. 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 Processor -- start 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 -- start start 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34102d40 0x7ffa34198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34112030 con 0x7ffa34103660 2024-09-12T02:16:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34111eb0 con 0x7ffa34102d40 2024-09-12T02:16:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa388ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341121b0 con 0x7ffa3410c170 2024-09-12T02:16:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa2bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50196/0 (socket says 172.21.15.3:50196) 2024-09-12T02:16:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.106+0000 7ffa2bfff640 1 -- 172.21.15.3:0/3584086711 learned_addr learned my addr 172.21.15.3:0/3584086711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa3259c640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34102d40 0x7ffa34198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa32d9d640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 -- 172.21.15.3:0/3584086711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 msgr2=0x7ffa3419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 -- 172.21.15.3:0/3584086711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34102d40 msgr2=0x7ffa34198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34102d40 0x7ffa34198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 -- 172.21.15.3:0/3584086711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3419d5c0 con 0x7ffa34103660 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa32d9d640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2bfff640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b3f0 tx=0x7ffa1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c015020 con 0x7ffa34103660 2024-09-12T02:16:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3419d8b0 con 0x7ffa34103660 2024-09-12T02:16:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c011d10 con 0x7ffa34103660 2024-09-12T02:16:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00f2e0 con 0x7ffa34103660 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.107+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a5190 con 0x7ffa34103660 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.108+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa341031a0 con 0x7ffa34103660 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.109+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ffa1c018020 con 0x7ffa34103660 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.110+0000 7ffa2b7fe640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 0x7ffa200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.110+0000 7ffa3259c640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 0x7ffa200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.110+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ffa1c0ce550 con 0x7ffa34103660 2024-09-12T02:16:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.110+0000 7ffa3259c640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 0x7ffa200a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffa240096c0 tx=0x7ffa24031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.114+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa1c01f020 con 0x7ffa34103660 2024-09-12T02:16:25.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:25.289+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffa34107210 con 0x7ffa34103660 2024-09-12T02:16:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:25 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:25 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:16:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:16:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:25 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:16:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.070+0000 7ffa2b7fe640 1 -- 172.21.15.3:0/3584086711 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7ffa1c09b8b0 con 0x7ffa34103660 2024-09-12T02:16:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-09-12T02:16:26.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 msgr2=0x7ffa200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 0x7ffa200a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffa240096c0 tx=0x7ffa24031490 comp rx=0 tx=0).stop 2024-09-12T02:16:26.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 msgr2=0x7ffa34198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:26.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b3f0 tx=0x7ffa1c00b8b0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 shutdown_connections 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa200a2b80 0x7ffa200a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa3410c170 0x7ffa3419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa34103660 0x7ffa34198b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 --2- 172.21.15.3:0/3584086711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa34102d40 0x7ffa34198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 >> 172.21.15.3:0/3584086711 conn(0x7ffa340fc7a0 msgr2=0x7ffa340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 shutdown_connections 2024-09-12T02:16:26.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:26.073+0000 7ffa388ce640 1 -- 172.21.15.3:0/3584086711 wait complete. 2024-09-12T02:16:26.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T02:16:27.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:26 smithi026 ceph-mon[35496]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:27.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:26 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:16:27.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:26 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:16:27.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:26 smithi026 ceph-mon[35496]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:16:27.022 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:26 smithi003 ceph-mon[31476]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:26 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:16:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:26 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:16:27.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:26 smithi003 ceph-mon[31476]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:16:27.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:26 smithi092 ceph-mon[36089]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:27.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:26 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:16:27.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:26 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3584086711' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:16:27.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:26 smithi092 ceph-mon[36089]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:16:27.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 -- 172.21.15.3:0/1819770909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 msgr2=0x7f89481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 --2- 172.21.15.3:0/1819770909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f89481021c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8938009900 tx=0x7f893802a410 comp rx=0 tx=0).stop 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 -- 172.21.15.3:0/1819770909 shutdown_connections 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 --2- 172.21.15.3:0/1819770909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 --2- 172.21.15.3:0/1819770909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 0x7f89481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 --2- 172.21.15.3:0/1819770909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f89481021c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 -- 172.21.15.3:0/1819770909 >> 172.21.15.3:0/1819770909 conn(0x7f89480fd8c0 msgr2=0x7f89480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 -- 172.21.15.3:0/1819770909 shutdown_connections 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.476+0000 7f894dc07640 1 -- 172.21.15.3:0/1819770909 wait complete. 2024-09-12T02:16:27.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 Processor -- start 2024-09-12T02:16:27.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 -- start start 2024-09-12T02:16:27.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 0x7f89481989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8948111f70 con 0x7f8948101de0 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8948111df0 con 0x7f8948106f10 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f894dc07640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89481120f0 con 0x7f8948102790 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f89477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f89477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50208/0 (socket says 172.21.15.3:50208) 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f89477fe640 1 -- 172.21.15.3:0/987928964 learned_addr learned my addr 172.21.15.3:0/987928964 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f8946ffd640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 0x7f89481989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f8947fff640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f89477fe640 1 -- 172.21.15.3:0/987928964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 msgr2=0x7f89481989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.477+0000 7f89477fe640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 0x7f89481989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f89477fe640 1 -- 172.21.15.3:0/987928964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 msgr2=0x7f894819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f89477fe640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f89477fe640 1 -- 172.21.15.3:0/987928964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894819d430 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f8947fff640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f89477fe640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8938006010 tx=0x7f89380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893803f030 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938003ce0 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894819d6c0 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938035070 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.478+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481a5000 con 0x7f8948101de0 2024-09-12T02:16:27.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.479+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89481021c0 con 0x7f8948101de0 2024-09-12T02:16:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.480+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8938042030 con 0x7f8948101de0 2024-09-12T02:16:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.481+0000 7f8944ff9640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 0x7f89300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.481+0000 7f8946ffd640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 0x7f89300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.481+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f89380ef5e0 con 0x7f8948101de0 2024-09-12T02:16:27.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.482+0000 7f8946ffd640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 0x7f89300a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89480fdd70 tx=0x7f893c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:27.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.484+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89380bc930 con 0x7f8948101de0 2024-09-12T02:16:27.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:27.661+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f894810a0f0 con 0x7f8948101de0 2024-09-12T02:16:28.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.099+0000 7f8944ff9640 1 -- 172.21.15.3:0/987928964 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f89380bc230 con 0x7f8948101de0 2024-09-12T02:16:28.100 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 msgr2=0x7f89300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 0x7f89300a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f89480fdd70 tx=0x7f893c00a250 comp rx=0 tx=0).stop 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 msgr2=0x7f8948198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8938006010 tx=0x7f89380042f0 comp rx=0 tx=0).stop 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 shutdown_connections 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f89300a2c00 0x7f89300a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:28.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8948106f10 0x7f894819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8948102790 0x7f89481989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 --2- 172.21.15.3:0/987928964 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8948101de0 0x7f8948198480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 >> 172.21.15.3:0/987928964 conn(0x7f89480fd8c0 msgr2=0x7f89480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 shutdown_connections 2024-09-12T02:16:28.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:28.102+0000 7f894dc07640 1 -- 172.21.15.3:0/987928964 wait complete. 2024-09-12T02:16:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:28 smithi026 ceph-mon[35496]: pgmap v272: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:28 smithi026 ceph-mon[35496]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:16:28.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:28 smithi026 ceph-mon[35496]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:16:28.422 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:28 smithi003 ceph-mon[31476]: pgmap v272: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:28.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:28 smithi003 ceph-mon[31476]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:16:28.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:28 smithi003 ceph-mon[31476]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:16:28.524 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:28 smithi092 ceph-mon[36089]: pgmap v272: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:28.525 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:28 smithi092 ceph-mon[36089]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:16:28.525 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:28 smithi092 ceph-mon[36089]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:16:28.753 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T02:16:29.008 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:29 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:16:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:29 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:16:29.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:29 smithi003 ceph-mon[31476]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:16:29.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:29 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:16:29.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:29 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:16:29.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:29 smithi026 ceph-mon[35496]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:16:29.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3035182106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102ef0 msgr2=0x7f70e410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3035182106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102ef0 0x7f70e410da40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009ca0 tx=0x7f70dc02a950 comp rx=0 tx=0).stop 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3035182106 shutdown_connections 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3035182106 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102ef0 0x7f70e410da40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3035182106 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102570 0x7f70e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3035182106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4100e20 0x7f70e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3035182106 >> 172.21.15.3:0/3035182106 conn(0x7f70e40fc7d0 msgr2=0x7f70e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3035182106 shutdown_connections 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.496+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3035182106 wait complete. 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 Processor -- start 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 -- start start 2024-09-12T02:16:29.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4100e20 0x7f70e4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 0x7f70e41a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4110f80 con 0x7f70e4102570 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4110e00 con 0x7f70e4102ef0 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70ea6b6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4111100 con 0x7f70e4100e20 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47306/0 (socket says 172.21.15.3:47306) 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70dbfff640 1 -- 172.21.15.3:0/3989838932 learned_addr learned my addr 172.21.15.3:0/3989838932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70e3fff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4100e20 0x7f70e4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.497+0000 7f70e8c53640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 0x7f70e41a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 -- 172.21.15.3:0/3989838932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4100e20 msgr2=0x7f70e4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:29.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4100e20 0x7f70e4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 -- 172.21.15.3:0/3989838932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 msgr2=0x7f70e41a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 0x7f70e41a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 -- 172.21.15.3:0/3989838932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e41a4830 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70e8c53640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 0x7f70e41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70dbfff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c890 tx=0x7f70cc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc004220 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc0043c0 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e41a4a00 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc004a00 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.498+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41a4f10 con 0x7f70e4102570 2024-09-12T02:16:29.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.499+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e4101200 con 0x7f70e4102570 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.500+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f70cc0079b0 con 0x7f70e4102570 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.501+0000 7f70e1ffb640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 0x7f70d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.501+0000 7f70e3fff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 0x7f70d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.501+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f70cc0ceb90 con 0x7f70e4102570 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.501+0000 7f70e3fff640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 0x7f70d00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70d400abe0 tx=0x7f70d400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:29.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.505+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70cc014030 con 0x7f70e4102570 2024-09-12T02:16:29.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:29 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:16:29.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:29 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/987928964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:16:29.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:29 smithi092 ceph-mon[36089]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:16:29.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:29.717+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f70e406a560 con 0x7f70e4102570 2024-09-12T02:16:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.114+0000 7f70e1ffb640 1 -- 172.21.15.3:0/3989838932 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f70cc09bc00 con 0x7f70e4102570 2024-09-12T02:16:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T02:16:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 msgr2=0x7f70d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 0x7f70d00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70d400abe0 tx=0x7f70d400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 msgr2=0x7f70e41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c890 tx=0x7f70cc00cc90 comp rx=0 tx=0).stop 2024-09-12T02:16:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 shutdown_connections 2024-09-12T02:16:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70d00a2b80 0x7f70d00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4102ef0 0x7f70e41a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102570 0x7f70e41997b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 --2- 172.21.15.3:0/3989838932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4100e20 0x7f70e4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 >> 172.21.15.3:0/3989838932 conn(0x7f70e40fc7d0 msgr2=0x7f70e41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 shutdown_connections 2024-09-12T02:16:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:30.117+0000 7f70ea6b6640 1 -- 172.21.15.3:0/3989838932 wait complete. 2024-09-12T02:16:30.440 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:30 smithi003 ceph-mon[31476]: pgmap v275: 97 pgs: 1 creating+activating, 8 creating+peering, 82 unknown, 6 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:30.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:30 smithi003 ceph-mon[31476]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:16:30.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:30 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:16:30.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:30 smithi003 ceph-mon[31476]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:16:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:30 smithi026 ceph-mon[35496]: pgmap v275: 97 pgs: 1 creating+activating, 8 creating+peering, 82 unknown, 6 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:30 smithi026 ceph-mon[35496]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:16:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:30 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:16:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:30 smithi026 ceph-mon[35496]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:16:30.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:30 smithi092 ceph-mon[36089]: pgmap v275: 97 pgs: 1 creating+activating, 8 creating+peering, 82 unknown, 6 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:16:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:30 smithi092 ceph-mon[36089]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:16:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:30 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:16:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:30 smithi092 ceph-mon[36089]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:16:30.780 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:16:31.041 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:31.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:31 smithi003 ceph-mon[31476]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:16:31.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:31 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:16:31.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:31 smithi003 ceph-mon[31476]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:16:31.403 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:31 smithi003 ceph-mon[31476]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 -- 172.21.15.3:0/480681924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 msgr2=0x7fef2410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 --2- 172.21.15.3:0/480681924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef2410d950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009930 tx=0x7fef1c02a420 comp rx=0 tx=0).stop 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 -- 172.21.15.3:0/480681924 shutdown_connections 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 --2- 172.21.15.3:0/480681924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef2410d950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 --2- 172.21.15.3:0/480681924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 0x7fef24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 --2- 172.21.15.3:0/480681924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 0x7fef24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 -- 172.21.15.3:0/480681924 >> 172.21.15.3:0/480681924 conn(0x7fef240fc7a0 msgr2=0x7fef240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 -- 172.21.15.3:0/480681924 shutdown_connections 2024-09-12T02:16:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 -- 172.21.15.3:0/480681924 wait complete. 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.484+0000 7fef294e1640 1 Processor -- start 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 -- start start 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 0x7fef240fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 0x7fef240fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef24110e80 con 0x7fef24101420 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef24110d00 con 0x7fef24100a80 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef294e1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef24111000 con 0x7fef24106a80 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47332/0 (socket says 172.21.15.3:47332) 2024-09-12T02:16:31.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 -- 172.21.15.3:0/1747359974 learned_addr learned my addr 172.21.15.3:0/1747359974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef237fe640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 0x7fef240fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef22ffd640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 0x7fef240fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 -- 172.21.15.3:0/1747359974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 msgr2=0x7fef240fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 0x7fef240fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 -- 172.21.15.3:0/1747359974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 msgr2=0x7fef240fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 0x7fef240fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 -- 172.21.15.3:0/1747359974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241a6950 con 0x7fef24101420 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.485+0000 7fef227fc640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef1800c890 tx=0x7fef1800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.486+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18004220 con 0x7fef24101420 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.486+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180043c0 con 0x7fef24101420 2024-09-12T02:16:31.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.486+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241a6c40 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.486+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180049e0 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.486+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241a7180 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.487+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef24100ee0 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.488+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fef180079b0 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.489+0000 7fef0bfff640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 0x7fef0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.489+0000 7fef22ffd640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 0x7fef0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.489+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fef180cef50 con 0x7fef24101420 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.489+0000 7fef22ffd640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 0x7fef0c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef140092f0 tx=0x7fef14009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:31.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.493+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef18014030 con 0x7fef24101420 2024-09-12T02:16:31.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:31 smithi026 ceph-mon[35496]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:16:31.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:31 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:16:31.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:31 smithi026 ceph-mon[35496]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:16:31.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:31 smithi026 ceph-mon[35496]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:16:31.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:31 smithi092 ceph-mon[36089]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:16:31.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:31 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3989838932' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:16:31.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:31 smithi092 ceph-mon[36089]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:16:31.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:31 smithi092 ceph-mon[36089]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:16:31.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.670+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef2410a000 con 0x7fef24101420 2024-09-12T02:16:31.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.671+0000 7fef0bfff640 1 -- 172.21.15.3:0/1747359974 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22101 (secure 0 0 0) 0x7fef1809c000 con 0x7fef24101420 2024-09-12T02:16:31.671 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:31.671 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:29.718438+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 msgr2=0x7fef0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 0x7fef0c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef140092f0 tx=0x7fef14009280 comp rx=0 tx=0).stop 2024-09-12T02:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 msgr2=0x7fef240fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef1800c890 tx=0x7fef1800cd50 comp rx=0 tx=0).stop 2024-09-12T02:16:31.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 shutdown_connections 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef0c0a2b80 0x7fef0c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef24106a80 0x7fef240fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fef24101420 0x7fef240fd320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 --2- 172.21.15.3:0/1747359974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef24100a80 0x7fef240fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 >> 172.21.15.3:0/1747359974 conn(0x7fef240fc7a0 msgr2=0x7fef2406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 shutdown_connections 2024-09-12T02:16:31.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:31.673+0000 7fef294e1640 1 -- 172.21.15.3:0/1747359974 wait complete. 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: pgmap v278: 97 pgs: 1 creating+activating, 16 creating+peering, 42 unknown, 38 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: Cluster is now healthy 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:active 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.343 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:32 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1747359974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:32.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: pgmap v278: 97 pgs: 1 creating+activating, 16 creating+peering, 42 unknown, 38 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: Cluster is now healthy 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:active 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:32 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1747359974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: pgmap v278: 97 pgs: 1 creating+activating, 16 creating+peering, 42 unknown, 38 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: Cluster is now healthy 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] up:active 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:32 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1747359974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:32.632 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:33.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 -- 172.21.15.3:0/3674540615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0106f10 msgr2=0x7fabc010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 --2- 172.21.15.3:0/3674540615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0106f10 0x7fabc010ea90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009ca0 tx=0x7fabb802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 -- 172.21.15.3:0/3674540615 shutdown_connections 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 --2- 172.21.15.3:0/3674540615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0106f10 0x7fabc010ea90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 --2- 172.21.15.3:0/3674540615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0102790 0x7fabc01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 --2- 172.21.15.3:0/3674540615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0101de0 0x7fabc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 -- 172.21.15.3:0/3674540615 >> 172.21.15.3:0/3674540615 conn(0x7fabc00fd8c0 msgr2=0x7fabc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 -- 172.21.15.3:0/3674540615 shutdown_connections 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.076+0000 7fabc8547640 1 -- 172.21.15.3:0/3674540615 wait complete. 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 Processor -- start 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 -- start start 2024-09-12T02:16:33.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 0x7fabc01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0106f10 0x7fabc019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0111f70 con 0x7fabc0102790 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0111df0 con 0x7fabc0106f10 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc8547640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fabc01120f0 con 0x7fabc0101de0 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc5ae2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc5ae2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47340/0 (socket says 172.21.15.3:47340) 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc5ae2640 1 -- 172.21.15.3:0/630048639 learned_addr learned my addr 172.21.15.3:0/630048639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc62e3640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 0x7fabc01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc6ae4640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0106f10 0x7fabc019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc5ae2640 1 -- 172.21.15.3:0/630048639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 msgr2=0x7fabc01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.077+0000 7fabc5ae2640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 0x7fabc01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc5ae2640 1 -- 172.21.15.3:0/630048639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0106f10 msgr2=0x7fabc019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc5ae2640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0106f10 0x7fabc019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc5ae2640 1 -- 172.21.15.3:0/630048639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc019d3a0 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc62e3640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 0x7fabc01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc5ae2640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d770 tx=0x7fabb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0017070 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc019d690 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb0005500 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0004e00 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.078+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01a4f70 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.079+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc01021c0 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.080+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fabb0004fa0 con 0x7fabc0102790 2024-09-12T02:16:33.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.081+0000 7fabb77fe640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 0x7fabac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:33.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.081+0000 7fabc62e3640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 0x7fabac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:33.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.081+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fabb00ceda0 con 0x7fabc0102790 2024-09-12T02:16:33.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.081+0000 7fabc62e3640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 0x7fabac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0058c0 tx=0x7fabbc005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:33.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.084+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabb009be50 con 0x7fabc0102790 2024-09-12T02:16:33.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.263+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabc010a0f0 con 0x7fabc0102790 2024-09-12T02:16:33.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.264+0000 7fabb77fe640 1 -- 172.21.15.3:0/630048639 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22101 (secure 0 0 0) 0x7fabb009b750 con 0x7fabc0102790 2024-09-12T02:16:33.265 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:33.266 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":81,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:29.718438+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:16:33.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.266+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 msgr2=0x7fabac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.266+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 0x7fabac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0058c0 tx=0x7fabbc005ec0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.266+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 msgr2=0x7fabc0198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.266+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabb000d770 tx=0x7fabb000dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 shutdown_connections 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fabac0a2b80 0x7fabac0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc0106f10 0x7fabc019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fabc0102790 0x7fabc0198930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 --2- 172.21.15.3:0/630048639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc0101de0 0x7fabc01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 >> 172.21.15.3:0/630048639 conn(0x7fabc00fd8c0 msgr2=0x7fabc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 shutdown_connections 2024-09-12T02:16:33.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:33.267+0000 7fabc8547640 1 -- 172.21.15.3:0/630048639 wait complete. 2024-09-12T02:16:33.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T02:16:34.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:34.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:34 smithi003 ceph-mon[31476]: pgmap v279: 97 pgs: 1 creating+activating, 8 creating+peering, 8 unknown, 80 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-09-12T02:16:34.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:34 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/630048639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:34 smithi026 ceph-mon[35496]: pgmap v279: 97 pgs: 1 creating+activating, 8 creating+peering, 8 unknown, 80 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-09-12T02:16:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:34 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/630048639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:34 smithi092 ceph-mon[36089]: pgmap v279: 97 pgs: 1 creating+activating, 8 creating+peering, 8 unknown, 80 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-09-12T02:16:34.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:34 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/630048639' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 -- 172.21.15.3:0/1117507615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 msgr2=0x7f975810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1117507615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975810ea90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9750009ca0 tx=0x7f975002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 -- 172.21.15.3:0/1117507615 shutdown_connections 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1117507615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975810ea90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1117507615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 0x7f97581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1117507615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 0x7f97581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 -- 172.21.15.3:0/1117507615 >> 172.21.15.3:0/1117507615 conn(0x7f97580fd8c0 msgr2=0x7f97580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 -- 172.21.15.3:0/1117507615 shutdown_connections 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.671+0000 7f975cbd7640 1 -- 172.21.15.3:0/1117507615 wait complete. 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 Processor -- start 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 -- start start 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 0x7f9758198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 0x7f9758198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9758111fb0 con 0x7f9758106f10 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9758111e30 con 0x7f9758101de0 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975cbd7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9758112130 con 0x7f9758102790 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47352/0 (socket says 172.21.15.3:47352) 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 -- 172.21.15.3:0/1952070450 learned_addr learned my addr 172.21.15.3:0/1952070450 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f975659c640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 0x7f9758198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f974ffff640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 0x7f9758198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 -- 172.21.15.3:0/1952070450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 msgr2=0x7f9758198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 0x7f9758198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 -- 172.21.15.3:0/1952070450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 msgr2=0x7f9758198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 0x7f9758198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.672+0000 7f9756d9d640 1 -- 172.21.15.3:0/1952070450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975819d4c0 con 0x7f9758106f10 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f9756d9d640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9750031a20 tx=0x7f9750031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750003c30 con 0x7f9758106f10 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975819d750 con 0x7f9758106f10 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9750003dd0 con 0x7f9758106f10 2024-09-12T02:16:34.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975002fcf0 con 0x7f9758106f10 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.673+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581a5090 con 0x7f9758106f10 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.674+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97581021c0 con 0x7f9758106f10 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.675+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9750004050 con 0x7f9758106f10 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.676+0000 7f974f7fe640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 0x7f97440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.676+0000 7f975659c640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 0x7f97440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.676+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f97500f04a0 con 0x7f9758106f10 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.676+0000 7f975659c640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 0x7f97440a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9748009890 tx=0x7f9748008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:34.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.679+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97500bd5e0 con 0x7f9758106f10 2024-09-12T02:16:34.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:34.890+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f975810a0f0 con 0x7f9758106f10 2024-09-12T02:16:35.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.121+0000 7f974f7fe640 1 -- 172.21.15.3:0/1952070450 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f975002fe90 con 0x7f9758106f10 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.123+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 msgr2=0x7f97440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.123+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 0x7f97440a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9748009890 tx=0x7f9748008040 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.123+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 msgr2=0x7f975819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.123+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9750031a20 tx=0x7f9750031a50 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 shutdown_connections 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97440a2b80 0x7f97440a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9758106f10 0x7f975819cde0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9758102790 0x7f9758198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 --2- 172.21.15.3:0/1952070450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9758101de0 0x7f9758198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 >> 172.21.15.3:0/1952070450 conn(0x7f97580fd8c0 msgr2=0x7f97580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 shutdown_connections 2024-09-12T02:16:35.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:35.124+0000 7f975cbd7640 1 -- 172.21.15.3:0/1952070450 wait complete. 2024-09-12T02:16:35.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:35 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:16:35.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:35 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:16:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:35 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:16:35.757 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:16:35.757 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-12T02:16:36.016 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:36.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:36 smithi003 ceph-mon[31476]: pgmap v280: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:16:36.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:36 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:16:36.280 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:36 smithi003 ceph-mon[31476]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.479+0000 7fd814446640 1 -- 172.21.15.3:0/387219264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 msgr2=0x7fd80c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 --2- 172.21.15.3:0/387219264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c10d950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd804009cd0 tx=0x7fd80402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 -- 172.21.15.3:0/387219264 shutdown_connections 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 --2- 172.21.15.3:0/387219264 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c10d950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 --2- 172.21.15.3:0/387219264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c100a80 0x7fd80c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 --2- 172.21.15.3:0/387219264 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c106a80 0x7fd80c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 -- 172.21.15.3:0/387219264 >> 172.21.15.3:0/387219264 conn(0x7fd80c0fc7a0 msgr2=0x7fd80c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 -- 172.21.15.3:0/387219264 shutdown_connections 2024-09-12T02:16:36.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 -- 172.21.15.3:0/387219264 wait complete. 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 Processor -- start 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.480+0000 7fd814446640 1 -- start start 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 0x7fd80c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 0x7fd80c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111fa0 con 0x7fd80c101420 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111e20 con 0x7fd80c106a80 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd814446640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c112120 con 0x7fd80c100a80 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47366/0 (socket says 172.21.15.3:47366) 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 -- 172.21.15.3:0/335800138 learned_addr learned my addr 172.21.15.3:0/335800138 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8129e3640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 0x7fd80c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8121e2640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 0x7fd80c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 -- 172.21.15.3:0/335800138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 msgr2=0x7fd80c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 0x7fd80c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 -- 172.21.15.3:0/335800138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 msgr2=0x7fd80c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 0x7fd80c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8119e1640 1 -- 172.21.15.3:0/335800138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c19d360 con 0x7fd80c101420 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8121e2640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 0x7fd80c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.481+0000 7fd8129e3640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 0x7fd80c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd8119e1640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c920 tx=0x7fd7fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc007c70 con 0x7fd80c101420 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7fc007e10 con 0x7fd80c101420 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c19d650 con 0x7fd80c101420 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc018890 con 0x7fd80c101420 2024-09-12T02:16:36.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.482+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1a4f30 con 0x7fd80c101420 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.483+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c100ee0 con 0x7fd80c101420 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.485+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd7fc018a50 con 0x7fd80c101420 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.485+0000 7fd8037fe640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 0x7fd7f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.485+0000 7fd8121e2640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 0x7fd7f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.485+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd7fc0cf390 con 0x7fd80c101420 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.486+0000 7fd8121e2640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 0x7fd7f80a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd808005fd0 tx=0x7fd80800b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:36.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.489+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7fc014030 con 0x7fd80c101420 2024-09-12T02:16:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:36 smithi026 ceph-mon[35496]: pgmap v280: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:16:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:36 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:16:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:36 smithi026 ceph-mon[35496]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:36 smithi092 ceph-mon[36089]: pgmap v280: 97 pgs: 97 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:16:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:36 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1952070450' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:16:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:36 smithi092 ceph-mon[36089]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:36.539 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:16:36.079+0000 7f5cd084a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T02:16:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:36.700+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd80c1a5260 con 0x7fd80c101420 2024-09-12T02:16:37.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.128+0000 7fd8037fe640 1 -- 172.21.15.3:0/335800138 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fd7fc09c4d0 con 0x7fd80c101420 2024-09-12T02:16:37.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.131+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 msgr2=0x7fd7f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.131+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 0x7fd7f80a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd808005fd0 tx=0x7fd80800b5d0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.131+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 msgr2=0x7fd80c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:37.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.131+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00c920 tx=0x7fd7fc00cde0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 shutdown_connections 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7f80a2b80 0x7fd7f80a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c106a80 0x7fd80c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd80c101420 0x7fd80c1988f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 --2- 172.21.15.3:0/335800138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd80c100a80 0x7fd80c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 >> 172.21.15.3:0/335800138 conn(0x7fd80c0fc7a0 msgr2=0x7fd80c0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 shutdown_connections 2024-09-12T02:16:37.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:37.132+0000 7fd814446640 1 -- 172.21.15.3:0/335800138 wait complete. 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-12T02:16:37.433 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T02:16:37.434 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-12T02:16:37.434 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T02:16:37.434 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:37 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-12T02:16:37.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T02:16:37.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:37 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-12T02:16:37.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:37 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:16:37.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T02:16:38.042 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-12T02:16:38.456 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:16:38.457 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:38 smithi003 ceph-mon[31476]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2024-09-12T02:16:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:16:38.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:38 smithi026 ceph-mon[35496]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:16:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: pgmap v281: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:16:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:16:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/335800138' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:16:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-12T02:16:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8, 5, 9]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:16:38.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:38 smithi092 ceph-mon[36089]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 -- 172.21.15.3:0/3889197071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa64101b40 msgr2=0x7faa6410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 --2- 172.21.15.3:0/3889197071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa64101b40 0x7faa6410da00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009ca0 tx=0x7faa5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 -- 172.21.15.3:0/3889197071 shutdown_connections 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 --2- 172.21.15.3:0/3889197071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa64101b40 0x7faa6410da00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 --2- 172.21.15.3:0/3889197071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa641011a0 0x7faa64101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 --2- 172.21.15.3:0/3889197071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa64106c30 0x7faa64107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 -- 172.21.15.3:0/3889197071 >> 172.21.15.3:0/3889197071 conn(0x7faa64077210 msgr2=0x7faa640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 -- 172.21.15.3:0/3889197071 shutdown_connections 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 -- 172.21.15.3:0/3889197071 wait complete. 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.553+0000 7faa69b19640 1 Processor -- start 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 -- start start 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 0x7faa64199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa64106c30 0x7faa641a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa64110fc0 con 0x7faa641011a0 2024-09-12T02:16:38.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faa64110e40 con 0x7faa64106c30 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa69b19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faa64111140 con 0x7faa64101b40 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38134/0 (socket says 172.21.15.3:38134) 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 -- 172.21.15.3:0/3486863335 learned_addr learned my addr 172.21.15.3:0/3486863335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa62ffd640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 0x7faa64199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 -- 172.21.15.3:0/3486863335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 msgr2=0x7faa64199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 0x7faa64199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 -- 172.21.15.3:0/3486863335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa64106c30 msgr2=0x7faa641a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa64106c30 0x7faa641a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:38.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa637fe640 1 -- 172.21.15.3:0/3486863335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa641a4a10 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.554+0000 7faa62ffd640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 0x7faa64199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa637fe640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54017070 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54009c30 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa641a4be0 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5400f6b0 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.555+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641a50f0 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.556+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa64101600 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.557+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faa540109d0 con 0x7faa641011a0 2024-09-12T02:16:38.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.558+0000 7faa60ff9640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 0x7faa4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.558+0000 7faa62ffd640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 0x7faa4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.558+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7faa540cedb0 con 0x7faa641011a0 2024-09-12T02:16:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.558+0000 7faa62ffd640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 0x7faa4c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa64100be0 tx=0x7faa58008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:38.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.562+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa540d3070 con 0x7faa641011a0 2024-09-12T02:16:38.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:38.774+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7faa641a52c0 con 0x7faa641011a0 2024-09-12T02:16:39.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.155+0000 7faa60ff9640 1 -- 172.21.15.3:0/3486863335 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7faa54009de0 con 0x7faa641011a0 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.157+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 msgr2=0x7faa4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.157+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 0x7faa4c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faa64100be0 tx=0x7faa58008040 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.157+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 msgr2=0x7faa64199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.157+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa5400d950 tx=0x7faa5400de10 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 shutdown_connections 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa4c0a2b80 0x7faa4c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa64106c30 0x7faa641a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa64101b40 0x7faa64199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 --2- 172.21.15.3:0/3486863335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa641011a0 0x7faa64199450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 >> 172.21.15.3:0/3486863335 conn(0x7faa64077210 msgr2=0x7faa64103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:39.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 shutdown_connections 2024-09-12T02:16:39.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:39.158+0000 7faa69b19640 1 -- 172.21.15.3:0/3486863335 wait complete. 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:active 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:16:39.452 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:39 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:active 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:16:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:39 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] up:active 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:16:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:39 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:16:39.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T02:16:40.074 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:40.481 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:active 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:16:40.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: Cluster is now healthy 2024-09-12T02:16:40.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 4 up:standby-replay 4 up:standby 2024-09-12T02:16:40.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-12T02:16:40.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:40 smithi003 ceph-mon[31476]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:active 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:16:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: Cluster is now healthy 2024-09-12T02:16:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 4 up:standby-replay 4 up:standby 2024-09-12T02:16:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-12T02:16:40.516 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:40 smithi026 ceph-mon[35496]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 5 op/s 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3486863335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:16:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] up:active 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: Cluster is now healthy 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 4 up:standby-replay 4 up:standby 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-12T02:16:40.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:40 smithi092 ceph-mon[36089]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.566+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/3323171514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4101de0 msgr2=0x7f7bc41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.566+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/3323171514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4101de0 0x7f7bc41021c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009900 tx=0x7f7bb402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/3323171514 shutdown_connections 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/3323171514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 0x7f7bc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/3323171514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102790 0x7f7bc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/3323171514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4101de0 0x7f7bc41021c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/3323171514 >> 172.21.15.3:0/3323171514 conn(0x7f7bc40fd8c0 msgr2=0x7f7bc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/3323171514 shutdown_connections 2024-09-12T02:16:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/3323171514 wait complete. 2024-09-12T02:16:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 Processor -- start 2024-09-12T02:16:40.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.567+0000 7f7bc9dbe640 1 -- start start 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101de0 0x7f7bc40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 0x7f7bc4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4111f70 con 0x7f7bc4102790 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4111df0 con 0x7f7bc4106f10 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc9dbe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41120f0 con 0x7f7bc4101de0 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38156/0 (socket says 172.21.15.3:38156) 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 -- 172.21.15.3:0/549947207 learned_addr learned my addr 172.21.15.3:0/549947207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc37fe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101de0 0x7f7bc40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc3fff640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 0x7f7bc4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 -- 172.21.15.3:0/549947207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101de0 msgr2=0x7f7bc40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101de0 0x7f7bc40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 -- 172.21.15.3:0/549947207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 msgr2=0x7f7bc4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 0x7f7bc4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 -- 172.21.15.3:0/549947207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc41a6890 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.568+0000 7f7bc2ffd640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00eab0 tx=0x7f7bbc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.569+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00ce40 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.569+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bbc011950 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.569+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc41a6b80 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.569+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc010ae0 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.569+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41a70c0 con 0x7f7bc4102790 2024-09-12T02:16:40.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.570+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc41021c0 con 0x7f7bc4102790 2024-09-12T02:16:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.571+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7bbc011b00 con 0x7f7bc4102790 2024-09-12T02:16:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.571+0000 7f7bc0ff9640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 0x7f7bac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.572+0000 7f7bc37fe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 0x7f7bac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.572+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7bbc0cf6d0 con 0x7f7bc4102790 2024-09-12T02:16:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.572+0000 7f7bc37fe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 0x7f7bac0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009230 tx=0x7f7bb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:40.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.576+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bbc016030 con 0x7f7bc4102790 2024-09-12T02:16:40.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:40.790+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7bc410a0f0 con 0x7f7bc4102790 2024-09-12T02:16:41.539 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:16:41.080+0000 7f5cd084a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T02:16:41.539 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:16:41.080+0000 7f5cd084a640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-12T02:16:41.539 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:16:41.080+0000 7f5cd084a640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-12T02:16:41.539 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:16:41.080+0000 7f5cd084a640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-12T02:16:41.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.742+0000 7f7bc0ff9640 1 -- 172.21.15.3:0/549947207 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f7bbc09c5c0 con 0x7f7bc4102790 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 msgr2=0x7f7bac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 0x7f7bac0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009230 tx=0x7f7bb40058d0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 msgr2=0x7f7bc4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00eab0 tx=0x7f7bbc00ef70 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 shutdown_connections 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bac0a2b80 0x7f7bac0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4106f10 0x7f7bc4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc4102790 0x7f7bc4074a30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 --2- 172.21.15.3:0/549947207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101de0 0x7f7bc40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:41.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.745+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 >> 172.21.15.3:0/549947207 conn(0x7f7bc40fd8c0 msgr2=0x7f7bc40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:41.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.746+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 shutdown_connections 2024-09-12T02:16:41.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:41.746+0000 7f7bc9dbe640 1 -- 172.21.15.3:0/549947207 wait complete. 2024-09-12T02:16:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:41 smithi026 ceph-mon[35496]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:41 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:active 2024-09-12T02:16:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:41 smithi026 ceph-mon[35496]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-12T02:16:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:41 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:41 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:16:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:41 smithi003 ceph-mon[31476]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:41 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:active 2024-09-12T02:16:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:41 smithi003 ceph-mon[31476]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-12T02:16:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:41 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:41 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:16:42.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-mon[36089]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 2.7 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:42.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] up:active 2024-09-12T02:16:42.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-mon[36089]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-12T02:16:42.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:42.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:41 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:16:42.423 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T02:16:42.704 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T02:16:42.976 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:43.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:42 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:16:43.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:42 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:43.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:42 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:16:43.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:42 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:43.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:42 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/549947207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:16:43.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:42 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:16:43.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 -- 172.21.15.3:0/3528768481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd4102fc0 msgr2=0x7f7cd41005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 --2- 172.21.15.3:0/3528768481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd4102fc0 0x7f7cd41005d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009900 tx=0x7f7cc402a400 comp rx=0 tx=0).stop 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 -- 172.21.15.3:0/3528768481 shutdown_connections 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 --2- 172.21.15.3:0/3528768481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd4110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 --2- 172.21.15.3:0/3528768481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd4102fc0 0x7f7cd41005d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 --2- 172.21.15.3:0/3528768481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 0x7f7cd4100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 -- 172.21.15.3:0/3528768481 >> 172.21.15.3:0/3528768481 conn(0x7f7cd4077210 msgr2=0x7f7cd40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 -- 172.21.15.3:0/3528768481 shutdown_connections 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 -- 172.21.15.3:0/3528768481 wait complete. 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.441+0000 7f7cdc285640 1 Processor -- start 2024-09-12T02:16:43.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 -- start start 2024-09-12T02:16:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 0x7f7cd419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4114180 con 0x7f7cd419ef10 2024-09-12T02:16:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4114000 con 0x7f7cd410e810 2024-09-12T02:16:43.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cdc285640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4114300 con 0x7f7cd4102b50 2024-09-12T02:16:43.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38166/0 (socket says 172.21.15.3:38166) 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 -- 172.21.15.3:0/986142699 learned_addr learned my addr 172.21.15.3:0/986142699 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda021640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 0x7f7cd419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cd9820640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 -- 172.21.15.3:0/986142699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 msgr2=0x7f7cd419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 0x7f7cd419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 -- 172.21.15.3:0/986142699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 msgr2=0x7f7cd419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 -- 172.21.15.3:0/986142699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd419f9d0 con 0x7f7cd419ef10 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cd9820640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:43.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.442+0000 7f7cda822640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00eb10 tx=0x7f7ccc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.443+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00ce20 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.443+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ccc005020 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.443+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd419fcc0 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.443+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc010690 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.443+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41a7450 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.444+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd4100090 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.445+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7ccc005540 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.446+0000 7f7ccb7fe640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 0x7f7cc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.446+0000 7f7cda021640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 0x7f7cc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.446+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7ccc0cf680 con 0x7f7cd419ef10 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.446+0000 7f7cda021640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 0x7f7cc00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000abe0 tx=0x7f7cd000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:43.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.450+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ccc09c570 con 0x7f7cd419ef10 2024-09-12T02:16:43.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.650+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7cd4103140 con 0x7f7cc00a2b80 2024-09-12T02:16:43.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.674+0000 7f7ccb7fe640 1 -- 172.21.15.3:0/986142699 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7cd4103140 con 0x7f7cc00a2b80 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 msgr2=0x7f7cc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 0x7f7cc00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000abe0 tx=0x7f7cd000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 msgr2=0x7f7cd419f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00eb10 tx=0x7f7ccc00ef10 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 shutdown_connections 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc00a2b80 0x7f7cc00a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd419ef10 0x7f7cd419f3c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd410e810 0x7f7cd419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 --2- 172.21.15.3:0/986142699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd4102b50 0x7f7cd419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 >> 172.21.15.3:0/986142699 conn(0x7f7cd4077210 msgr2=0x7f7cd410fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:43.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 shutdown_connections 2024-09-12T02:16:43.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:43.677+0000 7f7cdc285640 1 -- 172.21.15.3:0/986142699 wait complete. 2024-09-12T02:16:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:43 smithi026 ceph-mon[35496]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-mon[31476]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:44.028 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:16:43.655+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:16:44.029 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:16:43.655+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:16:44.029 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:16:43.655+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:16:44.029 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:16:43.655+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:16:44.029 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:16:43 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:16:43.655+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:16:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:43 smithi092 ceph-mon[36089]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.5 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:44.430 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T02:16:44.694 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:45.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:44 smithi026 ceph-mon[35496]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:45.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:44 smithi003 ceph-mon[31476]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:45.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:44 smithi092 ceph-mon[36089]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 -- 172.21.15.3:0/2695481220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac10c180 msgr2=0x7ff9ac10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/2695481220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac10c180 0x7ff9ac10e610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009ca0 tx=0x7ff9a402a950 comp rx=0 tx=0).stop 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 -- 172.21.15.3:0/2695481220 shutdown_connections 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/2695481220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac10c180 0x7ff9ac10e610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/2695481220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac103700 0x7ff9ac10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/2695481220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac102de0 0x7ff9ac1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 -- 172.21.15.3:0/2695481220 >> 172.21.15.3:0/2695481220 conn(0x7ff9ac0fc7a0 msgr2=0x7ff9ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 -- 172.21.15.3:0/2695481220 shutdown_connections 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.167+0000 7ff9b29f1640 1 -- 172.21.15.3:0/2695481220 wait complete. 2024-09-12T02:16:45.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 Processor -- start 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 -- start start 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 0x7ff9ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac10c180 0x7ff9ac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.168+0000 7ff9b29f1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111fb0 con 0x7ff9ac102de0 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9b29f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111e30 con 0x7ff9ac103700 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9b29f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac112130 con 0x7ff9ac10c180 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38190/0 (socket says 172.21.15.3:38190) 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 -- 172.21.15.3:0/3000255202 learned_addr learned my addr 172.21.15.3:0/3000255202 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9ab7fe640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 0x7ff9ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 -- 172.21.15.3:0/3000255202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac10c180 msgr2=0x7ff9ac19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9b0f8e640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac10c180 0x7ff9ac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac10c180 0x7ff9ac19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 -- 172.21.15.3:0/3000255202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 msgr2=0x7ff9ac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 0x7ff9ac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 -- 172.21.15.3:0/3000255202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac19d400 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9ab7fe640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 0x7ff9ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9abfff640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00d950 tx=0x7ff99c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c017030 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99c004020 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac19d6f0 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0047a0 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.169+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a4fd0 con 0x7ff9ac102de0 2024-09-12T02:16:45.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.170+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac1031c0 con 0x7ff9ac102de0 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.171+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff99c0041d0 con 0x7ff9ac102de0 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.172+0000 7ff9a97fa640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 0x7ff9940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.172+0000 7ff9ab7fe640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 0x7ff9940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.172+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff99c0cec00 con 0x7ff9ac102de0 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.173+0000 7ff9ab7fe640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 0x7ff9940a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:45.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.177+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff99c09b9f0 con 0x7ff9ac102de0 2024-09-12T02:16:45.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.376+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff9ac062630 con 0x7ff9940a2c90 2024-09-12T02:16:45.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.403+0000 7ff9a97fa640 1 -- 172.21.15.3:0/3000255202 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff9ac062630 con 0x7ff9940a2c90 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 msgr2=0x7ff9940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 0x7ff9940a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0006fd0 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 msgr2=0x7ff9ac198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00d950 tx=0x7ff99c00de10 comp rx=0 tx=0).stop 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 shutdown_connections 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff9940a2c90 0x7ff9940a5150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9ac10c180 0x7ff9ac19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ac103700 0x7ff9ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 --2- 172.21.15.3:0/3000255202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9ac102de0 0x7ff9ac198450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 >> 172.21.15.3:0/3000255202 conn(0x7ff9ac0fc7a0 msgr2=0x7ff9ac0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 shutdown_connections 2024-09-12T02:16:45.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:45.405+0000 7ff9b29f1640 1 -- 172.21.15.3:0/3000255202 wait complete. 2024-09-12T02:16:46.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:45 smithi026 ceph-mon[35496]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:46.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:45 smithi026 ceph-mon[35496]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:16:46.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:45 smithi026 ceph-mon[35496]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:46.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:45 smithi003 ceph-mon[31476]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:46.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:45 smithi003 ceph-mon[31476]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:16:46.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:45 smithi003 ceph-mon[31476]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:46.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:45 smithi092 ceph-mon[36089]: pgmap v287: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 25 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:46.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:45 smithi092 ceph-mon[36089]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:16:46.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:45 smithi092 ceph-mon[36089]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:46.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T02:16:46.341 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:46.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 -- 172.21.15.3:0/2163861713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f0102df0 msgr2=0x7f29f01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 --2- 172.21.15.3:0/2163861713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f0102df0 0x7f29f01031d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009900 tx=0x7f29e002a410 comp rx=0 tx=0).stop 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 -- 172.21.15.3:0/2163861713 shutdown_connections 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 --2- 172.21.15.3:0/2163861713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f010c2f0 0x7f29f010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 --2- 172.21.15.3:0/2163861713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f01037a0 0x7f29f010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 --2- 172.21.15.3:0/2163861713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f0102df0 0x7f29f01031d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 -- 172.21.15.3:0/2163861713 >> 172.21.15.3:0/2163861713 conn(0x7f29f00fc7d0 msgr2=0x7f29f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 -- 172.21.15.3:0/2163861713 shutdown_connections 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.821+0000 7f29f6115640 1 -- 172.21.15.3:0/2163861713 wait complete. 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 Processor -- start 2024-09-12T02:16:46.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 -- start start 2024-09-12T02:16:46.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 0x7f29f0100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f01037a0 0x7f29f0100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112190 con 0x7f29f010c2f0 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112010 con 0x7f29f01037a0 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29f6115640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112310 con 0x7f29f0102df0 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38208/0 (socket says 172.21.15.3:38208) 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 -- 172.21.15.3:0/651180583 learned_addr learned my addr 172.21.15.3:0/651180583 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:46.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29ef7fe640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 0x7f29f0100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29e6ffd640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f01037a0 0x7f29f0100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 -- 172.21.15.3:0/651180583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 msgr2=0x7f29f0100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 0x7f29f0100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 -- 172.21.15.3:0/651180583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f01037a0 msgr2=0x7f29f0100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f01037a0 0x7f29f0100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.822+0000 7f29effff640 1 -- 172.21.15.3:0/651180583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f01a6b50 con 0x7f29f010c2f0 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29ef7fe640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 0x7f29f0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29effff640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b970 tx=0x7f29e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8019070 con 0x7f29f010c2f0 2024-09-12T02:16:46.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e800cb40 con 0x7f29f010c2f0 2024-09-12T02:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01a6e40 con 0x7f29f010c2f0 2024-09-12T02:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8007650 con 0x7f29f010c2f0 2024-09-12T02:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.823+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a72f0 con 0x7f29f010c2f0 2024-09-12T02:16:46.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.824+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f006a790 con 0x7f29f010c2f0 2024-09-12T02:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.825+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f29e801c070 con 0x7f29f010c2f0 2024-09-12T02:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.826+0000 7f29ed7fa640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 0x7f29dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.826+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f29e80ce750 con 0x7f29f010c2f0 2024-09-12T02:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.826+0000 7f29ef7fe640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 0x7f29dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:46.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.827+0000 7f29ef7fe640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 0x7f29dc0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005ab0 tx=0x7f29e0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:46.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:46.830+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29e809b5f0 con 0x7f29f010c2f0 2024-09-12T02:16:47.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.032+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f29f0062630 con 0x7f29dc0a2c10 2024-09-12T02:16:47.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.036+0000 7f29ed7fa640 1 -- 172.21.15.3:0/651180583 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f29f0062630 con 0x7f29dc0a2c10 2024-09-12T02:16:47.037 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_0/e8cba0b2-2fbd-459a-b973-eafd9b95bd66 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 msgr2=0x7f29dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 0x7f29dc0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29e0005ab0 tx=0x7f29e0005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 msgr2=0x7f29f01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29e800b970 tx=0x7f29e800be30 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 shutdown_connections 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f29dc0a2c10 0x7f29dc0a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29f010c2f0 0x7f29f01014d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29f01037a0 0x7f29f0100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 --2- 172.21.15.3:0/651180583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29f0102df0 0x7f29f0100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 >> 172.21.15.3:0/651180583 conn(0x7f29f00fc7d0 msgr2=0x7f29f00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:47.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 shutdown_connections 2024-09-12T02:16:47.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:47.039+0000 7f29f6115640 1 -- 172.21.15.3:0/651180583 wait complete. 2024-09-12T02:16:47.838 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T02:16:48.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:47 smithi026 ceph-mon[35496]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:48.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:47 smithi026 ceph-mon[35496]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:48.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:47 smithi003 ceph-mon[31476]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:47 smithi003 ceph-mon[31476]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:48.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:47 smithi092 ceph-mon[36089]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 26 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:48.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:47 smithi092 ceph-mon[36089]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:48.096 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:48.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/1816340346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074eb0 msgr2=0x7fbbc0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/1816340346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074eb0 0x7fbbc0112650 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009900 tx=0x7fbbac02a480 comp rx=0 tx=0).stop 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/1816340346 shutdown_connections 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/1816340346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/1816340346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074eb0 0x7fbbc0112650 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/1816340346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074590 0x7fbbc0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/1816340346 >> 172.21.15.3:0/1816340346 conn(0x7fbbc00fd8c0 msgr2=0x7fbbc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/1816340346 shutdown_connections 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.545+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/1816340346 wait complete. 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 Processor -- start 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 -- start start 2024-09-12T02:16:48.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 0x7fbbc01a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc0118540 con 0x7fbbc0074590 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01183c0 con 0x7fbbc0074eb0 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc6ac3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc01186c0 con 0x7fbbc0112b90 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc485f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc485f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35408/0 (socket says 172.21.15.3:35408) 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc485f640 1 -- 172.21.15.3:0/3911719205 learned_addr learned my addr 172.21.15.3:0/3911719205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbbffff640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 0x7fbbc01a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.546+0000 7fbbc5060640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 -- 172.21.15.3:0/3911719205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 msgr2=0x7fbbc019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 -- 172.21.15.3:0/3911719205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 msgr2=0x7fbbc01a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 0x7fbbc01a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 -- 172.21.15.3:0/3911719205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc019f000 con 0x7fbbc0074590 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc5060640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbbffff640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 0x7fbbc01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc485f640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400d950 tx=0x7fbbb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4017030 con 0x7fbbc0074590 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb4004020 con 0x7fbbc0074590 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc019f2f0 con 0x7fbbc0074590 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40047a0 con 0x7fbbc0074590 2024-09-12T02:16:48.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.547+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01ab5e0 con 0x7fbbc0074590 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.548+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc0074970 con 0x7fbbc0074590 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.549+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbbb40041d0 con 0x7fbbc0074590 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.550+0000 7fbbbdffb640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 0x7fbbb00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.550+0000 7fbbbffff640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 0x7fbbb00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.550+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbbb40cec60 con 0x7fbbc0074590 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.550+0000 7fbbbffff640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 0x7fbbb00a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbbc01a00c0 tx=0x7fbbac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:48.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.554+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbb409b9f0 con 0x7fbbc0074590 2024-09-12T02:16:48.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.757+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbbc0105040 con 0x7fbbb00a2c90 2024-09-12T02:16:48.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.778+0000 7fbbbdffb640 1 -- 172.21.15.3:0/3911719205 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbbac0315b0 con 0x7fbbb00a2c90 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 msgr2=0x7fbbb00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 0x7fbbb00a5150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbbc01a00c0 tx=0x7fbbac005a20 comp rx=0 tx=0).stop 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 msgr2=0x7fbbc01a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400d950 tx=0x7fbbb400de10 comp rx=0 tx=0).stop 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 shutdown_connections 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbbb00a2c90 0x7fbbb00a5150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbc0112b90 0x7fbbc019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbc0074eb0 0x7fbbc01a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 --2- 172.21.15.3:0/3911719205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbbc0074590 0x7fbbc01a4710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 >> 172.21.15.3:0/3911719205 conn(0x7fbbc00fd8c0 msgr2=0x7fbbc00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 shutdown_connections 2024-09-12T02:16:48.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:48.780+0000 7fbbc6ac3640 1 -- 172.21.15.3:0/3911719205 wait complete. 2024-09-12T02:16:49.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T02:16:49.829 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:49.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:49 smithi003 ceph-mon[31476]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:49.951 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:49 smithi003 ceph-mon[31476]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:50.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:49 smithi026 ceph-mon[35496]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:50.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:49 smithi026 ceph-mon[35496]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:49 smithi092 ceph-mon[36089]: pgmap v289: 97 pgs: 97 active+clean; 591 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:49 smithi092 ceph-mon[36089]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 -- 172.21.15.3:0/4199481105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c101420 msgr2=0x7fb64c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 --2- 172.21.15.3:0/4199481105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c101420 0x7fb64c10d950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb644009cd0 tx=0x7fb64402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 -- 172.21.15.3:0/4199481105 shutdown_connections 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 --2- 172.21.15.3:0/4199481105 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c101420 0x7fb64c10d950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 --2- 172.21.15.3:0/4199481105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c100a80 0x7fb64c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 --2- 172.21.15.3:0/4199481105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c106a80 0x7fb64c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 -- 172.21.15.3:0/4199481105 >> 172.21.15.3:0/4199481105 conn(0x7fb64c0fc7a0 msgr2=0x7fb64c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 -- 172.21.15.3:0/4199481105 shutdown_connections 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 -- 172.21.15.3:0/4199481105 wait complete. 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.320+0000 7fb653f9f640 1 Processor -- start 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 -- start start 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c100a80 0x7fb64c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c101420 0x7fb64c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c110f70 con 0x7fb64c106a80 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c110df0 con 0x7fb64c101420 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb653f9f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c1110f0 con 0x7fb64c100a80 2024-09-12T02:16:50.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35430/0 (socket says 172.21.15.3:35430) 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 -- 172.21.15.3:0/2804171281 learned_addr learned my addr 172.21.15.3:0/2804171281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb651d3b640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c100a80 0x7fb64c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65153a640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c101420 0x7fb64c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 -- 172.21.15.3:0/2804171281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c100a80 msgr2=0x7fb64c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c100a80 0x7fb64c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 -- 172.21.15.3:0/2804171281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c101420 msgr2=0x7fb64c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c101420 0x7fb64c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 -- 172.21.15.3:0/2804171281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c19d520 con 0x7fb64c106a80 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.321+0000 7fb65253c640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb644009ca0 tx=0x7fb644003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.322+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644004240 con 0x7fb64c106a80 2024-09-12T02:16:50.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.322+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6440305a0 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.322+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c19d7b0 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.322+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64403a660 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.322+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1a50f0 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.323+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c100f60 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.324+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb644030750 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.324+0000 7fb642ffd640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 0x7fb6380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.325+0000 7fb651d3b640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 0x7fb6380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.325+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb64402e030 con 0x7fb64c106a80 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.325+0000 7fb651d3b640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 0x7fb6380a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb648009890 tx=0x7fb648008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:50.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.329+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6440bd2d0 con 0x7fb64c106a80 2024-09-12T02:16:50.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.530+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb64c062630 con 0x7fb6380a2c10 2024-09-12T02:16:50.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.535+0000 7fb642ffd640 1 -- 172.21.15.3:0/2804171281 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb64c062630 con 0x7fb6380a2c10 2024-09-12T02:16:50.536 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 msgr2=0x7fb6380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 0x7fb6380a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb648009890 tx=0x7fb648008040 comp rx=0 tx=0).stop 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 msgr2=0x7fb64c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb644009ca0 tx=0x7fb644003ba0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 shutdown_connections 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6380a2c10 0x7fb6380a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64c106a80 0x7fb64c19ce40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c101420 0x7fb64c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 --2- 172.21.15.3:0/2804171281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c100a80 0x7fb64c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:50.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 >> 172.21.15.3:0/2804171281 conn(0x7fb64c0fc7a0 msgr2=0x7fb64c06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:50.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 shutdown_connections 2024-09-12T02:16:50.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:50.537+0000 7fb653f9f640 1 -- 172.21.15.3:0/2804171281 wait complete. 2024-09-12T02:16:51.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:16:51.571 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:51.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:51 smithi003 ceph-mon[31476]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:51.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:51 smithi003 ceph-mon[31476]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:52.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:51 smithi026 ceph-mon[35496]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:51 smithi026 ceph-mon[35496]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:51 smithi092 ceph-mon[36089]: pgmap v290: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.4 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:16:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:51 smithi092 ceph-mon[36089]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 -- 172.21.15.3:0/3461130952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3410c170 msgr2=0x7fab3410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 --2- 172.21.15.3:0/3461130952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3410c170 0x7fab3410e610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ca0 tx=0x7fab2c02a950 comp rx=0 tx=0).stop 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 -- 172.21.15.3:0/3461130952 shutdown_connections 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 --2- 172.21.15.3:0/3461130952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab3410c170 0x7fab3410e610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 --2- 172.21.15.3:0/3461130952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34103660 0x7fab3410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 --2- 172.21.15.3:0/3461130952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab34102d40 0x7fab34103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 -- 172.21.15.3:0/3461130952 >> 172.21.15.3:0/3461130952 conn(0x7fab340fc7a0 msgr2=0x7fab340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 -- 172.21.15.3:0/3461130952 shutdown_connections 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.043+0000 7fab39bb8640 1 -- 172.21.15.3:0/3461130952 wait complete. 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 Processor -- start 2024-09-12T02:16:52.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 -- start start 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34102d40 0x7fab34198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 0x7fab3419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112030 con 0x7fab34103660 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111eb0 con 0x7fab3410c170 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab39bb8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab341121b0 con 0x7fab34102d40 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35442/0 (socket says 172.21.15.3:35442) 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab32ffd640 1 -- 172.21.15.3:0/263583856 learned_addr learned my addr 172.21.15.3:0/263583856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.044+0000 7fab337fe640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34102d40 0x7fab34198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab33fff640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 0x7fab3419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 -- 172.21.15.3:0/263583856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34102d40 msgr2=0x7fab34198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34102d40 0x7fab34198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 -- 172.21.15.3:0/263583856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 msgr2=0x7fab3419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 0x7fab3419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 -- 172.21.15.3:0/263583856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419d4e0 con 0x7fab34103660 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab33fff640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 0x7fab3419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab32ffd640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b3f0 tx=0x7fab1c00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c015020 con 0x7fab34103660 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419d7d0 con 0x7fab34103660 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c011d10 con 0x7fab34103660 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c010e60 con 0x7fab34103660 2024-09-12T02:16:52.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.045+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a50b0 con 0x7fab34103660 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.046+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab341031a0 con 0x7fab34103660 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.047+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fab1c018020 con 0x7fab34103660 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.048+0000 7fab30ff9640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 0x7fab200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.049+0000 7fab337fe640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 0x7fab200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.049+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fab1c0ce920 con 0x7fab34103660 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.049+0000 7fab337fe640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 0x7fab200a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab24009990 tx=0x7fab24009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:52.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.053+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab1c01f020 con 0x7fab34103660 2024-09-12T02:16:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.262+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab34107240 con 0x7fab34103660 2024-09-12T02:16:52.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.263+0000 7fab30ff9640 1 -- 172.21.15.3:0/263583856 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7fab1c09b820 con 0x7fab34103660 2024-09-12T02:16:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:52.264 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:16:52.264 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:16:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 msgr2=0x7fab200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 0x7fab200a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab24009990 tx=0x7fab24009110 comp rx=0 tx=0).stop 2024-09-12T02:16:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 msgr2=0x7fab34198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00b3f0 tx=0x7fab1c00b7f0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 shutdown_connections 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fab200a2c10 0x7fab200a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab3410c170 0x7fab3419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fab34103660 0x7fab34198a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 --2- 172.21.15.3:0/263583856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab34102d40 0x7fab34198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 >> 172.21.15.3:0/263583856 conn(0x7fab340fc7a0 msgr2=0x7fab340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 shutdown_connections 2024-09-12T02:16:52.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:52.265+0000 7fab39bb8640 1 -- 172.21.15.3:0/263583856 wait complete. 2024-09-12T02:16:52.891 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:16:53.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:52 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/263583856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:53.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:52 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/263583856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:53.151 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:53.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:52 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/263583856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:16:53.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1228606602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 msgr2=0x7f189c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:53.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1228606602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c10da40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1894009ca0 tx=0x7f189402a950 comp rx=0 tx=0).stop 2024-09-12T02:16:53.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1228606602 shutdown_connections 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1228606602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c10da40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1228606602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c102570 0x7f189c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1228606602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c100e20 0x7f189c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1228606602 >> 172.21.15.3:0/1228606602 conn(0x7f189c0fc7d0 msgr2=0x7f189c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1228606602 shutdown_connections 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.637+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1228606602 wait complete. 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 Processor -- start 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 -- start start 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 0x7f189c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c102570 0x7f189c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:53.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f189c110f80 con 0x7f189c102ef0 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f189c110e00 con 0x7f189c102570 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f18a0c4d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f189c111100 con 0x7f189c100e20 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35458/0 (socket says 172.21.15.3:35458) 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 -- 172.21.15.3:0/1143074980 learned_addr learned my addr 172.21.15.3:0/1143074980 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189a59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 0x7f189c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f1893fff640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c102570 0x7f189c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 -- 172.21.15.3:0/1143074980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 msgr2=0x7f189c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 0x7f189c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 -- 172.21.15.3:0/1143074980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c102570 msgr2=0x7f189c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c102570 0x7f189c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189ad9d640 1 -- 172.21.15.3:0/1143074980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a6950 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.638+0000 7f189a59c640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 0x7f189c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f189ad9d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f189402ae70 tx=0x7f189402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894034040 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a6be0 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894030920 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894045480 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.639+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a7110 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.640+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c101200 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.641+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1894037020 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.642+0000 7f18937fe640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 0x7f18880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.642+0000 7f189a59c640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 0x7f18880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.642+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f18940eff90 con 0x7f189c102ef0 2024-09-12T02:16:53.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.642+0000 7f189a59c640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 0x7f18880a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bfd0 tx=0x7f188c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:53.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.646+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18940bce90 con 0x7f189c102ef0 2024-09-12T02:16:53.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.823+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f189c10a0a0 con 0x7f189c102ef0 2024-09-12T02:16:53.824 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:53 smithi003 ceph-mon[31476]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-09-12T02:16:53.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.825+0000 7f18937fe640 1 -- 172.21.15.3:0/1143074980 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f18940bc790 con 0x7f189c102ef0 2024-09-12T02:16:53.826 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:53.826 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 msgr2=0x7f18880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 0x7f18880a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f188c00bfd0 tx=0x7f188c00b390 comp rx=0 tx=0).stop 2024-09-12T02:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 msgr2=0x7f189c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:53.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f189402ae70 tx=0x7f189402f5d0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 shutdown_connections 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c90 0x7f18880a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f189c102ef0 0x7f189c0fd700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f189c102570 0x7f189c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 --2- 172.21.15.3:0/1143074980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c100e20 0x7f189c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 >> 172.21.15.3:0/1143074980 conn(0x7f189c0fc7d0 msgr2=0x7f189c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 shutdown_connections 2024-09-12T02:16:53.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:53.827+0000 7f18a0c4d640 1 -- 172.21.15.3:0/1143074980 wait complete. 2024-09-12T02:16:54.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:53 smithi092 ceph-mon[36089]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-09-12T02:16:54.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:53 smithi026 ceph-mon[35496]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-09-12T02:16:54.466 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:16:54.466 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:16:54.723 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:55.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1143074980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:55.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1143074980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- 172.21.15.3:0/3027201206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060106a80 msgr2=0x7fe060106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 --2- 172.21.15.3:0/3027201206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060106a80 0x7fe060106e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe050009930 tx=0x7fe05002a420 comp rx=0 tx=0).stop 2024-09-12T02:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- 172.21.15.3:0/3027201206 shutdown_connections 2024-09-12T02:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 --2- 172.21.15.3:0/3027201206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060101420 0x7fe06010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 --2- 172.21.15.3:0/3027201206 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 0x7fe060100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 --2- 172.21.15.3:0/3027201206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060106a80 0x7fe060106e60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- 172.21.15.3:0/3027201206 >> 172.21.15.3:0/3027201206 conn(0x7fe0600fc7a0 msgr2=0x7fe0600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- 172.21.15.3:0/3027201206 shutdown_connections 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- 172.21.15.3:0/3027201206 wait complete. 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 Processor -- start 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.221+0000 7fe065c34640 1 -- start start 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 0x7fe0601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060106a80 0x7fe06019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe060111fe0 con 0x7fe060101420 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe060111e60 con 0x7fe060106a80 2024-09-12T02:16:55.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe065c34640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe060112160 con 0x7fe060100a80 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35488/0 (socket says 172.21.15.3:35488) 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 -- 172.21.15.3:0/2260806216 learned_addr learned my addr 172.21.15.3:0/2260806216 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05f7fe640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 0x7fe0601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05ffff640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060106a80 0x7fe06019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 -- 172.21.15.3:0/2260806216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 msgr2=0x7fe0601984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 0x7fe0601984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 -- 172.21.15.3:0/2260806216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060106a80 msgr2=0x7fe06019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060106a80 0x7fe06019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 -- 172.21.15.3:0/2260806216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06019d460 con 0x7fe060101420 2024-09-12T02:16:55.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.222+0000 7fe05effd640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d770 tx=0x7fe05400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.223+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054017070 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.223+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe054005500 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.223+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06019d750 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.223+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054004e00 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.223+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0601a5030 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.224+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe060100ee0 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.225+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe054004fa0 con 0x7fe060101420 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.225+0000 7fe05cff9640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 0x7fe0480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.226+0000 7fe05f7fe640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 0x7fe0480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:55.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.226+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe0540cebb0 con 0x7fe060101420 2024-09-12T02:16:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.226+0000 7fe05f7fe640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 0x7fe0480a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe050006010 tx=0x7fe050000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:55.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.230+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe05409b350 con 0x7fe060101420 2024-09-12T02:16:55.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1143074980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:16:55.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.403+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe060062630 con 0x7fe0480a2c10 2024-09-12T02:16:55.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.411+0000 7fe05cff9640 1 -- 172.21.15.3:0/2260806216 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7fe060062630 con 0x7fe0480a2c10 2024-09-12T02:16:55.414 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:55.416 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:16:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 msgr2=0x7fe0480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 0x7fe0480a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe050006010 tx=0x7fe050000bc0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 msgr2=0x7fe0601989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe05400d770 tx=0x7fe05400dc30 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 shutdown_connections 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe0480a2c10 0x7fe0480a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe060106a80 0x7fe06019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe060101420 0x7fe0601989f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 --2- 172.21.15.3:0/2260806216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe060100a80 0x7fe0601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 >> 172.21.15.3:0/2260806216 conn(0x7fe0600fc7a0 msgr2=0x7fe0600fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 shutdown_connections 2024-09-12T02:16:55.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:55.414+0000 7fe065c34640 1 -- 172.21.15.3:0/2260806216 wait complete. 2024-09-12T02:16:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:55 smithi003 ceph-mon[31476]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-12T02:16:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:55 smithi003 ceph-mon[31476]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:55 smithi092 ceph-mon[36089]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-12T02:16:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:55 smithi092 ceph-mon[36089]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:56.130 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-12T02:16:54.433319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638482,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":276,"num_read_kb":222,"num_write":408,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360380,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581444,"statfs":{"total":1151940427776,"available":1151571398656,"internally_reserved":0,"allocated":11390976,"data_stored":7168122,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42130,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":133,"num_read_kb":101,"num_write":72,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159326+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:16:38.159326+0000","last_peered":"2024-09-12T02:16:38.159326+0000","last_clean":"2024-09-12T02:16:38.159326+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:16:38.159326+0000","last_undegraded":"2024-09-12T02:16:38.159326+0000","last_fullsized":"2024-09-12T02:16:38.159326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139979+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:16:38.139979+0000","last_peered":"2024-09-12T02:16:38.139979+0000","last_clean":"2024-09-12T02:16:38.139979+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:16:38.139979+0000","last_undegraded":"2024-09-12T02:16:38.139979+0000","last_fullsized":"2024-09-12T02:16:38.139979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140008+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:16:38.140008+0000","last_peered":"2024-09-12T02:16:38.140008+0000","last_clean":"2024-09-12T02:16:38.140008+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:16:38.140008+0000","last_undegraded":"2024-09-12T02:16:38.140008+0000","last_fullsized":"2024-09-12T02:16:38.140008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158570+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:16:38.158570+0000","last_peered":"2024-09-12T02:16:38.158570+0000","last_clean":"2024-09-12T02:16:38.158570+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:16:38.158570+0000","last_undegraded":"2024-09-12T02:16:38.158570+0000","last_fullsized":"2024-09-12T02:16:38.158570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139897+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:16:38.139897+0000","last_peered":"2024-09-12T02:16:38.139897+0000","last_clean":"2024-09-12T02:16:38.139897+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:16:38.139897+0000","last_undegraded":"2024-09-12T02:16:38.139897+0000","last_fullsized":"2024-09-12T02:16:38.139897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156257+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:16:38.156257+0000","last_peered":"2024-09-12T02:16:38.156257+0000","last_clean":"2024-09-12T02:16:38.156257+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:16:38.156257+0000","last_undegraded":"2024-09-12T02:16:38.156257+0000","last_fullsized":"2024-09-12T02:16:38.156257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160088+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:16:38.160088+0000","last_peered":"2024-09-12T02:16:38.160088+0000","last_clean":"2024-09-12T02:16:38.160088+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:16:38.160088+0000","last_undegraded":"2024-09-12T02:16:38.160088+0000","last_fullsized":"2024-09-12T02:16:38.160088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137721+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:16:38.137721+0000","last_peered":"2024-09-12T02:16:38.137721+0000","last_clean":"2024-09-12T02:16:38.137721+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:16:38.137721+0000","last_undegraded":"2024-09-12T02:16:38.137721+0000","last_fullsized":"2024-09-12T02:16:38.137721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136274+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:16:38.136274+0000","last_peered":"2024-09-12T02:16:38.136274+0000","last_clean":"2024-09-12T02:16:38.136274+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:16:38.136274+0000","last_undegraded":"2024-09-12T02:16:38.136274+0000","last_fullsized":"2024-09-12T02:16:38.136274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157082+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:16:38.157082+0000","last_peered":"2024-09-12T02:16:38.157082+0000","last_clean":"2024-09-12T02:16:38.157082+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:16:38.157082+0000","last_undegraded":"2024-09-12T02:16:38.157082+0000","last_fullsized":"2024-09-12T02:16:38.157082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158697+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:16:38.158697+0000","last_peered":"2024-09-12T02:16:38.158697+0000","last_clean":"2024-09-12T02:16:38.158697+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:16:38.158697+0000","last_undegraded":"2024-09-12T02:16:38.158697+0000","last_fullsized":"2024-09-12T02:16:38.158697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157861+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:16:38.157861+0000","last_peered":"2024-09-12T02:16:38.157861+0000","last_clean":"2024-09-12T02:16:38.157861+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:16:38.157861+0000","last_undegraded":"2024-09-12T02:16:38.157861+0000","last_fullsized":"2024-09-12T02:16:38.157861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139935+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:16:38.139935+0000","last_peered":"2024-09-12T02:16:38.139935+0000","last_clean":"2024-09-12T02:16:38.139935+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:16:38.139935+0000","last_undegraded":"2024-09-12T02:16:38.139935+0000","last_fullsized":"2024-09-12T02:16:38.139935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139658+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:16:38.139658+0000","last_peered":"2024-09-12T02:16:38.139658+0000","last_clean":"2024-09-12T02:16:38.139658+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:16:38.139658+0000","last_undegraded":"2024-09-12T02:16:38.139658+0000","last_fullsized":"2024-09-12T02:16:38.139658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135680+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:16:38.135680+0000","last_peered":"2024-09-12T02:16:38.135680+0000","last_clean":"2024-09-12T02:16:38.135680+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:16:38.135680+0000","last_undegraded":"2024-09-12T02:16:38.135680+0000","last_fullsized":"2024-09-12T02:16:38.135680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159427+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:16:38.159427+0000","last_peered":"2024-09-12T02:16:38.159427+0000","last_clean":"2024-09-12T02:16:38.159427+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:16:38.159427+0000","last_undegraded":"2024-09-12T02:16:38.159427+0000","last_fullsized":"2024-09-12T02:16:38.159427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135750+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:16:38.135750+0000","last_peered":"2024-09-12T02:16:38.135750+0000","last_clean":"2024-09-12T02:16:38.135750+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:16:38.135750+0000","last_undegraded":"2024-09-12T02:16:38.135750+0000","last_fullsized":"2024-09-12T02:16:38.135750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142616+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:16:38.142616+0000","last_peered":"2024-09-12T02:16:38.142616+0000","last_clean":"2024-09-12T02:16:38.142616+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:16:38.142616+0000","last_undegraded":"2024-09-12T02:16:38.142616+0000","last_fullsized":"2024-09-12T02:16:38.142616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159081+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:16:38.159081+0000","last_peered":"2024-09-12T02:16:38.159081+0000","last_clean":"2024-09-12T02:16:38.159081+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:16:38.159081+0000","last_undegraded":"2024-09-12T02:16:38.159081+0000","last_fullsized":"2024-09-12T02:16:38.159081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157895+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:16:38.157895+0000","last_peered":"2024-09-12T02:16:38.157895+0000","last_clean":"2024-09-12T02:16:38.157895+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:16:38.157895+0000","last_undegraded":"2024-09-12T02:16:38.157895+0000","last_fullsized":"2024-09-12T02:16:38.157895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135816+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:16:38.135816+0000","last_peered":"2024-09-12T02:16:38.135816+0000","last_clean":"2024-09-12T02:16:38.135816+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:16:38.135816+0000","last_undegraded":"2024-09-12T02:16:38.135816+0000","last_fullsized":"2024-09-12T02:16:38.135816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138128+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:16:38.138128+0000","last_peered":"2024-09-12T02:16:38.138128+0000","last_clean":"2024-09-12T02:16:38.138128+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:16:38.138128+0000","last_undegraded":"2024-09-12T02:16:38.138128+0000","last_fullsized":"2024-09-12T02:16:38.138128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140366+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:16:38.140366+0000","last_peered":"2024-09-12T02:16:38.140366+0000","last_clean":"2024-09-12T02:16:38.140366+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:16:38.140366+0000","last_undegraded":"2024-09-12T02:16:38.140366+0000","last_fullsized":"2024-09-12T02:16:38.140366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138165+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:16:38.138165+0000","last_peered":"2024-09-12T02:16:38.138165+0000","last_clean":"2024-09-12T02:16:38.138165+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:16:38.138165+0000","last_undegraded":"2024-09-12T02:16:38.138165+0000","last_fullsized":"2024-09-12T02:16:38.138165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.143205+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:16:38.143205+0000","last_peered":"2024-09-12T02:16:38.143205+0000","last_clean":"2024-09-12T02:16:38.143205+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:16:38.143205+0000","last_undegraded":"2024-09-12T02:16:38.143205+0000","last_fullsized":"2024-09-12T02:16:38.143205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159095+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:16:38.159095+0000","last_peered":"2024-09-12T02:16:38.159095+0000","last_clean":"2024-09-12T02:16:38.159095+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:16:38.159095+0000","last_undegraded":"2024-09-12T02:16:38.159095+0000","last_fullsized":"2024-09-12T02:16:38.159095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156467+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:16:38.156467+0000","last_peered":"2024-09-12T02:16:38.156467+0000","last_clean":"2024-09-12T02:16:38.156467+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:16:38.156467+0000","last_undegraded":"2024-09-12T02:16:38.156467+0000","last_fullsized":"2024-09-12T02:16:38.156467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159557+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:16:38.159557+0000","last_peered":"2024-09-12T02:16:38.159557+0000","last_clean":"2024-09-12T02:16:38.159557+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:16:38.159557+0000","last_undegraded":"2024-09-12T02:16:38.159557+0000","last_fullsized":"2024-09-12T02:16:38.159557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142741+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:16:38.142741+0000","last_peered":"2024-09-12T02:16:38.142741+0000","last_clean":"2024-09-12T02:16:38.142741+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:16:38.142741+0000","last_undegraded":"2024-09-12T02:16:38.142741+0000","last_fullsized":"2024-09-12T02:16:38.142741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159777+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:16:38.159777+0000","last_peered":"2024-09-12T02:16:38.159777+0000","last_clean":"2024-09-12T02:16:38.159777+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:16:38.159777+0000","last_undegraded":"2024-09-12T02:16:38.159777+0000","last_fullsized":"2024-09-12T02:16:38.159777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158876+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:16:38.158876+0000","last_peered":"2024-09-12T02:16:38.158876+0000","last_clean":"2024-09-12T02:16:38.158876+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:16:38.158876+0000","last_undegraded":"2024-09-12T02:16:38.158876+0000","last_fullsized":"2024-09-12T02:16:38.158876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139696+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:16:38.139696+0000","last_peered":"2024-09-12T02:16:38.139696+0000","last_clean":"2024-09-12T02:16:38.139696+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:16:38.139696+0000","last_undegraded":"2024-09-12T02:16:38.139696+0000","last_fullsized":"2024-09-12T02:16:38.139696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:50.758675+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:16:50.758675+0000","last_peered":"2024-09-12T02:16:50.758675+0000","last_clean":"2024-09-12T02:16:50.758675+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:16:50.758675+0000","last_undegraded":"2024-09-12T02:16:50.758675+0000","last_fullsized":"2024-09-12T02:16:50.758675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140785+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:16:38.140785+0000","last_peered":"2024-09-12T02:16:38.140785+0000","last_clean":"2024-09-12T02:16:38.140785+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:16:38.140785+0000","last_undegraded":"2024-09-12T02:16:38.140785+0000","last_fullsized":"2024-09-12T02:16:38.140785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137776+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:16:38.137776+0000","last_peered":"2024-09-12T02:16:38.137776+0000","last_clean":"2024-09-12T02:16:38.137776+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:16:38.137776+0000","last_undegraded":"2024-09-12T02:16:38.137776+0000","last_fullsized":"2024-09-12T02:16:38.137776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135610+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:16:38.135610+0000","last_peered":"2024-09-12T02:16:38.135610+0000","last_clean":"2024-09-12T02:16:38.135610+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:16:38.135610+0000","last_undegraded":"2024-09-12T02:16:38.135610+0000","last_fullsized":"2024-09-12T02:16:38.135610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:48.755805+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:16:48.755805+0000","last_peered":"2024-09-12T02:16:48.755805+0000","last_clean":"2024-09-12T02:16:48.755805+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:16:48.755805+0000","last_undegraded":"2024-09-12T02:16:48.755805+0000","last_fullsized":"2024-09-12T02:16:48.755805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136233+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:16:38.136233+0000","last_peered":"2024-09-12T02:16:38.136233+0000","last_clean":"2024-09-12T02:16:38.136233+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:16:38.136233+0000","last_undegraded":"2024-09-12T02:16:38.136233+0000","last_fullsized":"2024-09-12T02:16:38.136233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140363+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:16:38.140363+0000","last_peered":"2024-09-12T02:16:38.140363+0000","last_clean":"2024-09-12T02:16:38.140363+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:16:38.140363+0000","last_undegraded":"2024-09-12T02:16:38.140363+0000","last_fullsized":"2024-09-12T02:16:38.140363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135783+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:16:38.135783+0000","last_peered":"2024-09-12T02:16:38.135783+0000","last_clean":"2024-09-12T02:16:38.135783+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:16:38.135783+0000","last_undegraded":"2024-09-12T02:16:38.135783+0000","last_fullsized":"2024-09-12T02:16:38.135783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'23","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:48.772266+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:16:48.772266+0000","last_peered":"2024-09-12T02:16:48.772266+0000","last_clean":"2024-09-12T02:16:48.772266+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:16:48.772266+0000","last_undegraded":"2024-09-12T02:16:48.772266+0000","last_fullsized":"2024-09-12T02:16:48.772266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":38,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140182+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:16:38.140182+0000","last_peered":"2024-09-12T02:16:38.140182+0000","last_clean":"2024-09-12T02:16:38.140182+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:16:38.140182+0000","last_undegraded":"2024-09-12T02:16:38.140182+0000","last_fullsized":"2024-09-12T02:16:38.140182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157126+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:16:38.157126+0000","last_peered":"2024-09-12T02:16:38.157126+0000","last_clean":"2024-09-12T02:16:38.157126+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:16:38.157126+0000","last_undegraded":"2024-09-12T02:16:38.157126+0000","last_fullsized":"2024-09-12T02:16:38.157126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160799+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:16:38.160799+0000","last_peered":"2024-09-12T02:16:38.160799+0000","last_clean":"2024-09-12T02:16:38.160799+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:16:38.160799+0000","last_undegraded":"2024-09-12T02:16:38.160799+0000","last_fullsized":"2024-09-12T02:16:38.160799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:50.762880+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:16:50.762880+0000","last_peered":"2024-09-12T02:16:50.762880+0000","last_clean":"2024-09-12T02:16:50.762880+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:16:50.762880+0000","last_undegraded":"2024-09-12T02:16:50.762880+0000","last_fullsized":"2024-09-12T02:16:50.762880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137650+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:16:38.137650+0000","last_peered":"2024-09-12T02:16:38.137650+0000","last_clean":"2024-09-12T02:16:38.137650+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:16:38.137650+0000","last_undegraded":"2024-09-12T02:16:38.137650+0000","last_fullsized":"2024-09-12T02:16:38.137650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142089+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:16:38.142089+0000","last_peered":"2024-09-12T02:16:38.142089+0000","last_clean":"2024-09-12T02:16:38.142089+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:16:38.142089+0000","last_undegraded":"2024-09-12T02:16:38.142089+0000","last_fullsized":"2024-09-12T02:16:38.142089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739199+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:16:39.739199+0000","last_peered":"2024-09-12T02:16:39.739199+0000","last_clean":"2024-09-12T02:16:39.739199+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:16:39.739199+0000","last_undegraded":"2024-09-12T02:16:39.739199+0000","last_fullsized":"2024-09-12T02:16:39.739199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159706+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:16:38.159706+0000","last_peered":"2024-09-12T02:16:38.159706+0000","last_clean":"2024-09-12T02:16:38.159706+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:16:38.159706+0000","last_undegraded":"2024-09-12T02:16:38.159706+0000","last_fullsized":"2024-09-12T02:16:38.159706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138164+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:16:38.138164+0000","last_peered":"2024-09-12T02:16:38.138164+0000","last_clean":"2024-09-12T02:16:38.138164+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:16:38.138164+0000","last_undegraded":"2024-09-12T02:16:38.138164+0000","last_fullsized":"2024-09-12T02:16:38.138164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:49.759219+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:16:49.759219+0000","last_peered":"2024-09-12T02:16:49.759219+0000","last_clean":"2024-09-12T02:16:49.759219+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:16:49.759219+0000","last_undegraded":"2024-09-12T02:16:49.759219+0000","last_fullsized":"2024-09-12T02:16:49.759219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138219+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:16:38.138219+0000","last_peered":"2024-09-12T02:16:38.138219+0000","last_clean":"2024-09-12T02:16:38.138219+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:16:38.138219+0000","last_undegraded":"2024-09-12T02:16:38.138219+0000","last_fullsized":"2024-09-12T02:16:38.138219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137830+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137830+0000","last_peered":"2024-09-12T02:16:38.137830+0000","last_clean":"2024-09-12T02:16:38.137830+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137830+0000","last_undegraded":"2024-09-12T02:16:38.137830+0000","last_fullsized":"2024-09-12T02:16:38.137830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138042+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:16:38.138042+0000","last_peered":"2024-09-12T02:16:38.138042+0000","last_clean":"2024-09-12T02:16:38.138042+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:16:38.138042+0000","last_undegraded":"2024-09-12T02:16:38.138042+0000","last_fullsized":"2024-09-12T02:16:38.138042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157616+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:16:38.157616+0000","last_peered":"2024-09-12T02:16:38.157616+0000","last_clean":"2024-09-12T02:16:38.157616+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:16:38.157616+0000","last_undegraded":"2024-09-12T02:16:38.157616+0000","last_fullsized":"2024-09-12T02:16:38.157616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:51.763485+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:16:51.763485+0000","last_peered":"2024-09-12T02:16:51.763485+0000","last_clean":"2024-09-12T02:16:51.763485+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:16:51.763485+0000","last_undegraded":"2024-09-12T02:16:51.763485+0000","last_fullsized":"2024-09-12T02:16:51.763485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140764+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:16:38.140764+0000","last_peered":"2024-09-12T02:16:38.140764+0000","last_clean":"2024-09-12T02:16:38.140764+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:16:38.140764+0000","last_undegraded":"2024-09-12T02:16:38.140764+0000","last_fullsized":"2024-09-12T02:16:38.140764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:51.763881+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:16:51.763881+0000","last_peered":"2024-09-12T02:16:51.763881+0000","last_clean":"2024-09-12T02:16:51.763881+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:16:51.763881+0000","last_undegraded":"2024-09-12T02:16:51.763881+0000","last_fullsized":"2024-09-12T02:16:51.763881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156353+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:16:38.156353+0000","last_peered":"2024-09-12T02:16:38.156353+0000","last_clean":"2024-09-12T02:16:38.156353+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:16:38.156353+0000","last_undegraded":"2024-09-12T02:16:38.156353+0000","last_fullsized":"2024-09-12T02:16:38.156353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156722+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:16:38.156722+0000","last_peered":"2024-09-12T02:16:38.156722+0000","last_clean":"2024-09-12T02:16:38.156722+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:16:38.156722+0000","last_undegraded":"2024-09-12T02:16:38.156722+0000","last_fullsized":"2024-09-12T02:16:38.156722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156574+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:16:38.156574+0000","last_peered":"2024-09-12T02:16:38.156574+0000","last_clean":"2024-09-12T02:16:38.156574+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:16:38.156574+0000","last_undegraded":"2024-09-12T02:16:38.156574+0000","last_fullsized":"2024-09-12T02:16:38.156574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:46.123850+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:46.123850+0000","last_peered":"2024-09-12T02:16:46.123850+0000","last_clean":"2024-09-12T02:16:46.123850+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:46.123850+0000","last_undegraded":"2024-09-12T02:16:46.123850+0000","last_fullsized":"2024-09-12T02:16:46.123850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138258+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:16:38.138258+0000","last_peered":"2024-09-12T02:16:38.138258+0000","last_clean":"2024-09-12T02:16:38.138258+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:16:38.138258+0000","last_undegraded":"2024-09-12T02:16:38.138258+0000","last_fullsized":"2024-09-12T02:16:38.138258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.161245+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:16:38.161245+0000","last_peered":"2024-09-12T02:16:38.161245+0000","last_clean":"2024-09-12T02:16:38.161245+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:16:38.161245+0000","last_undegraded":"2024-09-12T02:16:38.161245+0000","last_fullsized":"2024-09-12T02:16:38.161245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139845+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:16:38.139845+0000","last_peered":"2024-09-12T02:16:38.139845+0000","last_clean":"2024-09-12T02:16:38.139845+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:16:38.139845+0000","last_undegraded":"2024-09-12T02:16:38.139845+0000","last_fullsized":"2024-09-12T02:16:38.139845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159016+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:16:38.159016+0000","last_peered":"2024-09-12T02:16:38.159016+0000","last_clean":"2024-09-12T02:16:38.159016+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:16:38.159016+0000","last_undegraded":"2024-09-12T02:16:38.159016+0000","last_fullsized":"2024-09-12T02:16:38.159016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158385+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:16:38.158385+0000","last_peered":"2024-09-12T02:16:38.158385+0000","last_clean":"2024-09-12T02:16:38.158385+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:16:38.158385+0000","last_undegraded":"2024-09-12T02:16:38.158385+0000","last_fullsized":"2024-09-12T02:16:38.158385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137813+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:16:38.137813+0000","last_peered":"2024-09-12T02:16:38.137813+0000","last_clean":"2024-09-12T02:16:38.137813+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:16:38.137813+0000","last_undegraded":"2024-09-12T02:16:38.137813+0000","last_fullsized":"2024-09-12T02:16:38.137813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140393+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:16:38.140393+0000","last_peered":"2024-09-12T02:16:38.140393+0000","last_clean":"2024-09-12T02:16:38.140393+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:16:38.140393+0000","last_undegraded":"2024-09-12T02:16:38.140393+0000","last_fullsized":"2024-09-12T02:16:38.140393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135916+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:16:38.135916+0000","last_peered":"2024-09-12T02:16:38.135916+0000","last_clean":"2024-09-12T02:16:38.135916+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:16:38.135916+0000","last_undegraded":"2024-09-12T02:16:38.135916+0000","last_fullsized":"2024-09-12T02:16:38.135916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140265+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:16:38.140265+0000","last_peered":"2024-09-12T02:16:38.140265+0000","last_clean":"2024-09-12T02:16:38.140265+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:16:38.140265+0000","last_undegraded":"2024-09-12T02:16:38.140265+0000","last_fullsized":"2024-09-12T02:16:38.140265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138254+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:16:38.138254+0000","last_peered":"2024-09-12T02:16:38.138254+0000","last_clean":"2024-09-12T02:16:38.138254+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:16:38.138254+0000","last_undegraded":"2024-09-12T02:16:38.138254+0000","last_fullsized":"2024-09-12T02:16:38.138254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:48.759884+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:16:48.759884+0000","last_peered":"2024-09-12T02:16:48.759884+0000","last_clean":"2024-09-12T02:16:48.759884+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:16:48.759884+0000","last_undegraded":"2024-09-12T02:16:48.759884+0000","last_fullsized":"2024-09-12T02:16:48.759884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47880,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":150,"num_read_kb":113,"num_write":225,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":165710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":196,"ondisk_log_size":196,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1071009,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1031743,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438955,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438423,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444095,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":480760,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":480933,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:16:56.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:16:56.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:55 smithi026 ceph-mon[35496]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-12T02:16:56.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:55 smithi026 ceph-mon[35496]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:56.394 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- 172.21.15.3:0/1291533456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec101de0 msgr2=0x7f24ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 --2- 172.21.15.3:0/1291533456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec101de0 0x7f24ec1021c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009900 tx=0x7f24e002a410 comp rx=0 tx=0).stop 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- 172.21.15.3:0/1291533456 shutdown_connections 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 --2- 172.21.15.3:0/1291533456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec106f10 0x7f24ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 --2- 172.21.15.3:0/1291533456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 0x7f24ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 --2- 172.21.15.3:0/1291533456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec101de0 0x7f24ec1021c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- 172.21.15.3:0/1291533456 >> 172.21.15.3:0/1291533456 conn(0x7f24ec0fd8c0 msgr2=0x7f24ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:56.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- 172.21.15.3:0/1291533456 shutdown_connections 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- 172.21.15.3:0/1291533456 wait complete. 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 Processor -- start 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.902+0000 7f24f31a2640 1 -- start start 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec101de0 0x7f24ec198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 0x7f24ec1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec111f90 con 0x7f24ec106f10 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec111e10 con 0x7f24ec102790 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f31a2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec112110 con 0x7f24ec101de0 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35492/0 (socket says 172.21.15.3:35492) 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 -- 172.21.15.3:0/3415554594 learned_addr learned my addr 172.21.15.3:0/3415554594 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:56.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24ebfff640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 0x7f24ec1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f0f3e640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec101de0 0x7f24ec198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 -- 172.21.15.3:0/3415554594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec101de0 msgr2=0x7f24ec198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec101de0 0x7f24ec198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 -- 172.21.15.3:0/3415554594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 msgr2=0x7f24ec1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 0x7f24ec1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 -- 172.21.15.3:0/3415554594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec19d440 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24f173f640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b970 tx=0x7f24e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.903+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4019070 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.904+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24e400cb40 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.904+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec19d730 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.904+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4007650 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.904+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec1a5010 con 0x7f24ec106f10 2024-09-12T02:16:56.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.905+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ec1021c0 con 0x7f24ec106f10 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.906+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f24e401c070 con 0x7f24ec106f10 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.906+0000 7f24e9ffb640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 0x7f24dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.906+0000 7f24f0f3e640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 0x7f24dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.907+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f24e40ce630 con 0x7f24ec106f10 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.907+0000 7f24f0f3e640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 0x7f24dc0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:56.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:56.911+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24e409aee0 con 0x7f24ec106f10 2024-09-12T02:16:57.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.085+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f24ec062630 con 0x7f24dc0a2c10 2024-09-12T02:16:57.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.094+0000 7f24e9ffb640 1 -- 172.21.15.3:0/3415554594 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258983 (secure 0 0 0) 0x7f24ec062630 con 0x7f24dc0a2c10 2024-09-12T02:16:57.097 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:16:57.098 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:16:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 msgr2=0x7f24dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 0x7f24dc0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00058d0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 msgr2=0x7f24ec19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f24e400b970 tx=0x7f24e400be30 comp rx=0 tx=0).stop 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 shutdown_connections 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f24dc0a2c10 0x7f24dc0a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24ec106f10 0x7f24ec19cd60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24ec102790 0x7f24ec1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 --2- 172.21.15.3:0/3415554594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24ec101de0 0x7f24ec198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 >> 172.21.15.3:0/3415554594 conn(0x7f24ec0fd8c0 msgr2=0x7f24ec0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 shutdown_connections 2024-09-12T02:16:57.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:57.097+0000 7f24f31a2640 1 -- 172.21.15.3:0/3415554594 wait complete. 2024-09-12T02:16:57.827 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-12T02:16:56.433816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649517,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":308,"num_read_kb":267,"num_write":420,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360420,"kb_used_data":11164,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581404,"statfs":{"total":1151940427776,"available":1151571357696,"internally_reserved":0,"allocated":11431936,"data_stored":7205484,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51115,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":124,"num_write":37,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003228"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159326+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:16:38.159326+0000","last_peered":"2024-09-12T02:16:38.159326+0000","last_clean":"2024-09-12T02:16:38.159326+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:16:38.159326+0000","last_undegraded":"2024-09-12T02:16:38.159326+0000","last_fullsized":"2024-09-12T02:16:38.159326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139979+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:16:38.139979+0000","last_peered":"2024-09-12T02:16:38.139979+0000","last_clean":"2024-09-12T02:16:38.139979+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:16:38.139979+0000","last_undegraded":"2024-09-12T02:16:38.139979+0000","last_fullsized":"2024-09-12T02:16:38.139979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140008+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:16:38.140008+0000","last_peered":"2024-09-12T02:16:38.140008+0000","last_clean":"2024-09-12T02:16:38.140008+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:16:38.140008+0000","last_undegraded":"2024-09-12T02:16:38.140008+0000","last_fullsized":"2024-09-12T02:16:38.140008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158570+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:16:38.158570+0000","last_peered":"2024-09-12T02:16:38.158570+0000","last_clean":"2024-09-12T02:16:38.158570+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:16:38.158570+0000","last_undegraded":"2024-09-12T02:16:38.158570+0000","last_fullsized":"2024-09-12T02:16:38.158570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139897+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:16:38.139897+0000","last_peered":"2024-09-12T02:16:38.139897+0000","last_clean":"2024-09-12T02:16:38.139897+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:16:38.139897+0000","last_undegraded":"2024-09-12T02:16:38.139897+0000","last_fullsized":"2024-09-12T02:16:38.139897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156257+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:16:38.156257+0000","last_peered":"2024-09-12T02:16:38.156257+0000","last_clean":"2024-09-12T02:16:38.156257+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:16:38.156257+0000","last_undegraded":"2024-09-12T02:16:38.156257+0000","last_fullsized":"2024-09-12T02:16:38.156257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160088+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:16:38.160088+0000","last_peered":"2024-09-12T02:16:38.160088+0000","last_clean":"2024-09-12T02:16:38.160088+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:16:38.160088+0000","last_undegraded":"2024-09-12T02:16:38.160088+0000","last_fullsized":"2024-09-12T02:16:38.160088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137721+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:16:38.137721+0000","last_peered":"2024-09-12T02:16:38.137721+0000","last_clean":"2024-09-12T02:16:38.137721+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:16:38.137721+0000","last_undegraded":"2024-09-12T02:16:38.137721+0000","last_fullsized":"2024-09-12T02:16:38.137721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136274+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:16:38.136274+0000","last_peered":"2024-09-12T02:16:38.136274+0000","last_clean":"2024-09-12T02:16:38.136274+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:16:38.136274+0000","last_undegraded":"2024-09-12T02:16:38.136274+0000","last_fullsized":"2024-09-12T02:16:38.136274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157082+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:16:38.157082+0000","last_peered":"2024-09-12T02:16:38.157082+0000","last_clean":"2024-09-12T02:16:38.157082+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:16:38.157082+0000","last_undegraded":"2024-09-12T02:16:38.157082+0000","last_fullsized":"2024-09-12T02:16:38.157082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158697+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:16:38.158697+0000","last_peered":"2024-09-12T02:16:38.158697+0000","last_clean":"2024-09-12T02:16:38.158697+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:16:38.158697+0000","last_undegraded":"2024-09-12T02:16:38.158697+0000","last_fullsized":"2024-09-12T02:16:38.158697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157861+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:16:38.157861+0000","last_peered":"2024-09-12T02:16:38.157861+0000","last_clean":"2024-09-12T02:16:38.157861+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:16:38.157861+0000","last_undegraded":"2024-09-12T02:16:38.157861+0000","last_fullsized":"2024-09-12T02:16:38.157861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139935+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:16:38.139935+0000","last_peered":"2024-09-12T02:16:38.139935+0000","last_clean":"2024-09-12T02:16:38.139935+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:16:38.139935+0000","last_undegraded":"2024-09-12T02:16:38.139935+0000","last_fullsized":"2024-09-12T02:16:38.139935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139658+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:16:38.139658+0000","last_peered":"2024-09-12T02:16:38.139658+0000","last_clean":"2024-09-12T02:16:38.139658+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:16:38.139658+0000","last_undegraded":"2024-09-12T02:16:38.139658+0000","last_fullsized":"2024-09-12T02:16:38.139658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135680+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:16:38.135680+0000","last_peered":"2024-09-12T02:16:38.135680+0000","last_clean":"2024-09-12T02:16:38.135680+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:16:38.135680+0000","last_undegraded":"2024-09-12T02:16:38.135680+0000","last_fullsized":"2024-09-12T02:16:38.135680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159427+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:16:38.159427+0000","last_peered":"2024-09-12T02:16:38.159427+0000","last_clean":"2024-09-12T02:16:38.159427+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:16:38.159427+0000","last_undegraded":"2024-09-12T02:16:38.159427+0000","last_fullsized":"2024-09-12T02:16:38.159427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135750+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:16:38.135750+0000","last_peered":"2024-09-12T02:16:38.135750+0000","last_clean":"2024-09-12T02:16:38.135750+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:16:38.135750+0000","last_undegraded":"2024-09-12T02:16:38.135750+0000","last_fullsized":"2024-09-12T02:16:38.135750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142616+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:16:38.142616+0000","last_peered":"2024-09-12T02:16:38.142616+0000","last_clean":"2024-09-12T02:16:38.142616+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:16:38.142616+0000","last_undegraded":"2024-09-12T02:16:38.142616+0000","last_fullsized":"2024-09-12T02:16:38.142616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159081+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:16:38.159081+0000","last_peered":"2024-09-12T02:16:38.159081+0000","last_clean":"2024-09-12T02:16:38.159081+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:16:38.159081+0000","last_undegraded":"2024-09-12T02:16:38.159081+0000","last_fullsized":"2024-09-12T02:16:38.159081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157895+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:16:38.157895+0000","last_peered":"2024-09-12T02:16:38.157895+0000","last_clean":"2024-09-12T02:16:38.157895+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:16:38.157895+0000","last_undegraded":"2024-09-12T02:16:38.157895+0000","last_fullsized":"2024-09-12T02:16:38.157895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135816+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:16:38.135816+0000","last_peered":"2024-09-12T02:16:38.135816+0000","last_clean":"2024-09-12T02:16:38.135816+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:16:38.135816+0000","last_undegraded":"2024-09-12T02:16:38.135816+0000","last_fullsized":"2024-09-12T02:16:38.135816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138128+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:16:38.138128+0000","last_peered":"2024-09-12T02:16:38.138128+0000","last_clean":"2024-09-12T02:16:38.138128+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:16:38.138128+0000","last_undegraded":"2024-09-12T02:16:38.138128+0000","last_fullsized":"2024-09-12T02:16:38.138128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140366+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:16:38.140366+0000","last_peered":"2024-09-12T02:16:38.140366+0000","last_clean":"2024-09-12T02:16:38.140366+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:16:38.140366+0000","last_undegraded":"2024-09-12T02:16:38.140366+0000","last_fullsized":"2024-09-12T02:16:38.140366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138165+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:16:38.138165+0000","last_peered":"2024-09-12T02:16:38.138165+0000","last_clean":"2024-09-12T02:16:38.138165+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:16:38.138165+0000","last_undegraded":"2024-09-12T02:16:38.138165+0000","last_fullsized":"2024-09-12T02:16:38.138165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.143205+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:16:38.143205+0000","last_peered":"2024-09-12T02:16:38.143205+0000","last_clean":"2024-09-12T02:16:38.143205+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:16:38.143205+0000","last_undegraded":"2024-09-12T02:16:38.143205+0000","last_fullsized":"2024-09-12T02:16:38.143205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159095+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:16:38.159095+0000","last_peered":"2024-09-12T02:16:38.159095+0000","last_clean":"2024-09-12T02:16:38.159095+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:16:38.159095+0000","last_undegraded":"2024-09-12T02:16:38.159095+0000","last_fullsized":"2024-09-12T02:16:38.159095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156467+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:16:38.156467+0000","last_peered":"2024-09-12T02:16:38.156467+0000","last_clean":"2024-09-12T02:16:38.156467+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:16:38.156467+0000","last_undegraded":"2024-09-12T02:16:38.156467+0000","last_fullsized":"2024-09-12T02:16:38.156467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159557+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:16:38.159557+0000","last_peered":"2024-09-12T02:16:38.159557+0000","last_clean":"2024-09-12T02:16:38.159557+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:16:38.159557+0000","last_undegraded":"2024-09-12T02:16:38.159557+0000","last_fullsized":"2024-09-12T02:16:38.159557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142741+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:16:38.142741+0000","last_peered":"2024-09-12T02:16:38.142741+0000","last_clean":"2024-09-12T02:16:38.142741+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:16:38.142741+0000","last_undegraded":"2024-09-12T02:16:38.142741+0000","last_fullsized":"2024-09-12T02:16:38.142741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159777+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:16:38.159777+0000","last_peered":"2024-09-12T02:16:38.159777+0000","last_clean":"2024-09-12T02:16:38.159777+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:16:38.159777+0000","last_undegraded":"2024-09-12T02:16:38.159777+0000","last_fullsized":"2024-09-12T02:16:38.159777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158876+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:16:38.158876+0000","last_peered":"2024-09-12T02:16:38.158876+0000","last_clean":"2024-09-12T02:16:38.158876+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:16:38.158876+0000","last_undegraded":"2024-09-12T02:16:38.158876+0000","last_fullsized":"2024-09-12T02:16:38.158876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139696+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:16:38.139696+0000","last_peered":"2024-09-12T02:16:38.139696+0000","last_clean":"2024-09-12T02:16:38.139696+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:16:38.139696+0000","last_undegraded":"2024-09-12T02:16:38.139696+0000","last_fullsized":"2024-09-12T02:16:38.139696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:50.758675+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:16:50.758675+0000","last_peered":"2024-09-12T02:16:50.758675+0000","last_clean":"2024-09-12T02:16:50.758675+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:16:50.758675+0000","last_undegraded":"2024-09-12T02:16:50.758675+0000","last_fullsized":"2024-09-12T02:16:50.758675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140785+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:16:38.140785+0000","last_peered":"2024-09-12T02:16:38.140785+0000","last_clean":"2024-09-12T02:16:38.140785+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:16:38.140785+0000","last_undegraded":"2024-09-12T02:16:38.140785+0000","last_fullsized":"2024-09-12T02:16:38.140785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137776+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:16:38.137776+0000","last_peered":"2024-09-12T02:16:38.137776+0000","last_clean":"2024-09-12T02:16:38.137776+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:16:38.137776+0000","last_undegraded":"2024-09-12T02:16:38.137776+0000","last_fullsized":"2024-09-12T02:16:38.137776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135610+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:16:38.135610+0000","last_peered":"2024-09-12T02:16:38.135610+0000","last_clean":"2024-09-12T02:16:38.135610+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:16:38.135610+0000","last_undegraded":"2024-09-12T02:16:38.135610+0000","last_fullsized":"2024-09-12T02:16:38.135610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:53.762597+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:16:53.762597+0000","last_peered":"2024-09-12T02:16:53.762597+0000","last_clean":"2024-09-12T02:16:53.762597+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:16:53.762597+0000","last_undegraded":"2024-09-12T02:16:53.762597+0000","last_fullsized":"2024-09-12T02:16:53.762597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136233+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:16:38.136233+0000","last_peered":"2024-09-12T02:16:38.136233+0000","last_clean":"2024-09-12T02:16:38.136233+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:16:38.136233+0000","last_undegraded":"2024-09-12T02:16:38.136233+0000","last_fullsized":"2024-09-12T02:16:38.136233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140363+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:16:38.140363+0000","last_peered":"2024-09-12T02:16:38.140363+0000","last_clean":"2024-09-12T02:16:38.140363+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:16:38.140363+0000","last_undegraded":"2024-09-12T02:16:38.140363+0000","last_fullsized":"2024-09-12T02:16:38.140363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135783+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:16:38.135783+0000","last_peered":"2024-09-12T02:16:38.135783+0000","last_clean":"2024-09-12T02:16:38.135783+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:16:38.135783+0000","last_undegraded":"2024-09-12T02:16:38.135783+0000","last_fullsized":"2024-09-12T02:16:38.135783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:53.767797+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:16:53.767797+0000","last_peered":"2024-09-12T02:16:53.767797+0000","last_clean":"2024-09-12T02:16:53.767797+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:16:53.767797+0000","last_undegraded":"2024-09-12T02:16:53.767797+0000","last_fullsized":"2024-09-12T02:16:53.767797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":68,"num_write":30,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140182+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:16:38.140182+0000","last_peered":"2024-09-12T02:16:38.140182+0000","last_clean":"2024-09-12T02:16:38.140182+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:16:38.140182+0000","last_undegraded":"2024-09-12T02:16:38.140182+0000","last_fullsized":"2024-09-12T02:16:38.140182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157126+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:16:38.157126+0000","last_peered":"2024-09-12T02:16:38.157126+0000","last_clean":"2024-09-12T02:16:38.157126+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:16:38.157126+0000","last_undegraded":"2024-09-12T02:16:38.157126+0000","last_fullsized":"2024-09-12T02:16:38.157126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160799+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:16:38.160799+0000","last_peered":"2024-09-12T02:16:38.160799+0000","last_clean":"2024-09-12T02:16:38.160799+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:16:38.160799+0000","last_undegraded":"2024-09-12T02:16:38.160799+0000","last_fullsized":"2024-09-12T02:16:38.160799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:50.762880+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:16:50.762880+0000","last_peered":"2024-09-12T02:16:50.762880+0000","last_clean":"2024-09-12T02:16:50.762880+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:16:50.762880+0000","last_undegraded":"2024-09-12T02:16:50.762880+0000","last_fullsized":"2024-09-12T02:16:50.762880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137650+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:16:38.137650+0000","last_peered":"2024-09-12T02:16:38.137650+0000","last_clean":"2024-09-12T02:16:38.137650+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:16:38.137650+0000","last_undegraded":"2024-09-12T02:16:38.137650+0000","last_fullsized":"2024-09-12T02:16:38.137650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142089+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:16:38.142089+0000","last_peered":"2024-09-12T02:16:38.142089+0000","last_clean":"2024-09-12T02:16:38.142089+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:16:38.142089+0000","last_undegraded":"2024-09-12T02:16:38.142089+0000","last_fullsized":"2024-09-12T02:16:38.142089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739199+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:16:39.739199+0000","last_peered":"2024-09-12T02:16:39.739199+0000","last_clean":"2024-09-12T02:16:39.739199+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:16:39.739199+0000","last_undegraded":"2024-09-12T02:16:39.739199+0000","last_fullsized":"2024-09-12T02:16:39.739199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159706+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:16:38.159706+0000","last_peered":"2024-09-12T02:16:38.159706+0000","last_clean":"2024-09-12T02:16:38.159706+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:16:38.159706+0000","last_undegraded":"2024-09-12T02:16:38.159706+0000","last_fullsized":"2024-09-12T02:16:38.159706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138164+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:16:38.138164+0000","last_peered":"2024-09-12T02:16:38.138164+0000","last_clean":"2024-09-12T02:16:38.138164+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:16:38.138164+0000","last_undegraded":"2024-09-12T02:16:38.138164+0000","last_fullsized":"2024-09-12T02:16:38.138164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:54.764888+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:16:54.764888+0000","last_peered":"2024-09-12T02:16:54.764888+0000","last_clean":"2024-09-12T02:16:54.764888+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:16:54.764888+0000","last_undegraded":"2024-09-12T02:16:54.764888+0000","last_fullsized":"2024-09-12T02:16:54.764888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138219+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:16:38.138219+0000","last_peered":"2024-09-12T02:16:38.138219+0000","last_clean":"2024-09-12T02:16:38.138219+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:16:38.138219+0000","last_undegraded":"2024-09-12T02:16:38.138219+0000","last_fullsized":"2024-09-12T02:16:38.138219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137830+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137830+0000","last_peered":"2024-09-12T02:16:38.137830+0000","last_clean":"2024-09-12T02:16:38.137830+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137830+0000","last_undegraded":"2024-09-12T02:16:38.137830+0000","last_fullsized":"2024-09-12T02:16:38.137830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138042+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:16:38.138042+0000","last_peered":"2024-09-12T02:16:38.138042+0000","last_clean":"2024-09-12T02:16:38.138042+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:16:38.138042+0000","last_undegraded":"2024-09-12T02:16:38.138042+0000","last_fullsized":"2024-09-12T02:16:38.138042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157616+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:16:38.157616+0000","last_peered":"2024-09-12T02:16:38.157616+0000","last_clean":"2024-09-12T02:16:38.157616+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:16:38.157616+0000","last_undegraded":"2024-09-12T02:16:38.157616+0000","last_fullsized":"2024-09-12T02:16:38.157616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:51.763485+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:16:51.763485+0000","last_peered":"2024-09-12T02:16:51.763485+0000","last_clean":"2024-09-12T02:16:51.763485+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:16:51.763485+0000","last_undegraded":"2024-09-12T02:16:51.763485+0000","last_fullsized":"2024-09-12T02:16:51.763485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140764+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:16:38.140764+0000","last_peered":"2024-09-12T02:16:38.140764+0000","last_clean":"2024-09-12T02:16:38.140764+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:16:38.140764+0000","last_undegraded":"2024-09-12T02:16:38.140764+0000","last_fullsized":"2024-09-12T02:16:38.140764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:51.763881+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:16:51.763881+0000","last_peered":"2024-09-12T02:16:51.763881+0000","last_clean":"2024-09-12T02:16:51.763881+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:16:51.763881+0000","last_undegraded":"2024-09-12T02:16:51.763881+0000","last_fullsized":"2024-09-12T02:16:51.763881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156353+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:16:38.156353+0000","last_peered":"2024-09-12T02:16:38.156353+0000","last_clean":"2024-09-12T02:16:38.156353+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:16:38.156353+0000","last_undegraded":"2024-09-12T02:16:38.156353+0000","last_fullsized":"2024-09-12T02:16:38.156353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156722+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:16:38.156722+0000","last_peered":"2024-09-12T02:16:38.156722+0000","last_clean":"2024-09-12T02:16:38.156722+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:16:38.156722+0000","last_undegraded":"2024-09-12T02:16:38.156722+0000","last_fullsized":"2024-09-12T02:16:38.156722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156574+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:16:38.156574+0000","last_peered":"2024-09-12T02:16:38.156574+0000","last_clean":"2024-09-12T02:16:38.156574+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:16:38.156574+0000","last_undegraded":"2024-09-12T02:16:38.156574+0000","last_fullsized":"2024-09-12T02:16:38.156574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:53.124634+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:53.124634+0000","last_peered":"2024-09-12T02:16:53.124634+0000","last_clean":"2024-09-12T02:16:53.124634+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:53.124634+0000","last_undegraded":"2024-09-12T02:16:53.124634+0000","last_fullsized":"2024-09-12T02:16:53.124634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138258+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:16:38.138258+0000","last_peered":"2024-09-12T02:16:38.138258+0000","last_clean":"2024-09-12T02:16:38.138258+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:16:38.138258+0000","last_undegraded":"2024-09-12T02:16:38.138258+0000","last_fullsized":"2024-09-12T02:16:38.138258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.161245+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:16:38.161245+0000","last_peered":"2024-09-12T02:16:38.161245+0000","last_clean":"2024-09-12T02:16:38.161245+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:16:38.161245+0000","last_undegraded":"2024-09-12T02:16:38.161245+0000","last_fullsized":"2024-09-12T02:16:38.161245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139845+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:16:38.139845+0000","last_peered":"2024-09-12T02:16:38.139845+0000","last_clean":"2024-09-12T02:16:38.139845+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:16:38.139845+0000","last_undegraded":"2024-09-12T02:16:38.139845+0000","last_fullsized":"2024-09-12T02:16:38.139845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159016+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:16:38.159016+0000","last_peered":"2024-09-12T02:16:38.159016+0000","last_clean":"2024-09-12T02:16:38.159016+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:16:38.159016+0000","last_undegraded":"2024-09-12T02:16:38.159016+0000","last_fullsized":"2024-09-12T02:16:38.159016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158385+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:16:38.158385+0000","last_peered":"2024-09-12T02:16:38.158385+0000","last_clean":"2024-09-12T02:16:38.158385+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:16:38.158385+0000","last_undegraded":"2024-09-12T02:16:38.158385+0000","last_fullsized":"2024-09-12T02:16:38.158385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137813+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:16:38.137813+0000","last_peered":"2024-09-12T02:16:38.137813+0000","last_clean":"2024-09-12T02:16:38.137813+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:16:38.137813+0000","last_undegraded":"2024-09-12T02:16:38.137813+0000","last_fullsized":"2024-09-12T02:16:38.137813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140393+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:16:38.140393+0000","last_peered":"2024-09-12T02:16:38.140393+0000","last_clean":"2024-09-12T02:16:38.140393+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:16:38.140393+0000","last_undegraded":"2024-09-12T02:16:38.140393+0000","last_fullsized":"2024-09-12T02:16:38.140393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135916+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:16:38.135916+0000","last_peered":"2024-09-12T02:16:38.135916+0000","last_clean":"2024-09-12T02:16:38.135916+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:16:38.135916+0000","last_undegraded":"2024-09-12T02:16:38.135916+0000","last_fullsized":"2024-09-12T02:16:38.135916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140265+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:16:38.140265+0000","last_peered":"2024-09-12T02:16:38.140265+0000","last_clean":"2024-09-12T02:16:38.140265+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:16:38.140265+0000","last_undegraded":"2024-09-12T02:16:38.140265+0000","last_fullsized":"2024-09-12T02:16:38.140265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138254+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:16:38.138254+0000","last_peered":"2024-09-12T02:16:38.138254+0000","last_clean":"2024-09-12T02:16:38.138254+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:16:38.138254+0000","last_undegraded":"2024-09-12T02:16:38.138254+0000","last_fullsized":"2024-09-12T02:16:38.138254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":51,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:53.765918+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:16:53.765918+0000","last_peered":"2024-09-12T02:16:53.765918+0000","last_clean":"2024-09-12T02:16:53.765918+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:16:53.765918+0000","last_undegraded":"2024-09-12T02:16:53.765918+0000","last_fullsized":"2024-09-12T02:16:53.765918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58915,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":182,"num_read_kb":158,"num_write":237,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":185259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1082044,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1031743,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438955,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438423,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":491795,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":496225,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:16:57.832 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:16:57.832 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T02:16:57.833 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T02:16:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:57 smithi092 ceph-mon[36089]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-12T02:16:58.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:57 smithi092 ceph-mon[36089]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:58.091 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:16:58.213 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:57 smithi003 ceph-mon[31476]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-12T02:16:58.213 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:57 smithi003 ceph-mon[31476]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:57 smithi026 ceph-mon[35496]: pgmap v293: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-12T02:16:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:57 smithi026 ceph-mon[35496]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:16:58.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/406966879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a38102e00 msgr2=0x7f2a38103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/406966879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a38102e00 0x7f2a38103280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009900 tx=0x7f2a2c02a410 comp rx=0 tx=0).stop 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/406966879 shutdown_connections 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/406966879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a381037c0 0x7f2a3810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/406966879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a38102e00 0x7f2a38103280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/406966879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 0x7f2a38102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/406966879 >> 172.21.15.3:0/406966879 conn(0x7f2a380fd390 msgr2=0x7f2a380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/406966879 shutdown_connections 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/406966879 wait complete. 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.603+0000 7f2a3cd6f640 1 Processor -- start 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 -- start start 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 0x7f2a3819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a38102e00 0x7f2a3819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3810d880 con 0x7f2a381037c0 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3810d700 con 0x7f2a38102e00 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3cd6f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3810da00 con 0x7f2a38101c00 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56454/0 (socket says 172.21.15.3:56454) 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 -- 172.21.15.3:0/468777341 learned_addr learned my addr 172.21.15.3:0/468777341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a35d9b640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a38102e00 0x7f2a3819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a3659c640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 0x7f2a3819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 -- 172.21.15.3:0/468777341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 msgr2=0x7f2a3819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 0x7f2a3819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 -- 172.21.15.3:0/468777341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a38102e00 msgr2=0x7f2a3819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a38102e00 0x7f2a3819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 -- 172.21.15.3:0/468777341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a381a1d50 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a36d9d640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000eaa0 tx=0x7f2a3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.604+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000cdb0 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.605+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a30005020 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.605+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a381a1fe0 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.605+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300116b0 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.605+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a381a24a0 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.606+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a381a2730 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.607+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2a30005560 con 0x7f2a381037c0 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.608+0000 7f2a277fe640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 0x7f2a200a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:16:58.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.608+0000 7f2a3659c640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 0x7f2a200a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:16:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.608+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2a300cf410 con 0x7f2a381037c0 2024-09-12T02:16:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.608+0000 7f2a3659c640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 0x7f2a200a5100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a280058c0 tx=0x7f2a2800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:16:58.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.612+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a30016030 con 0x7f2a381037c0 2024-09-12T02:16:58.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.861+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2a381a2b60 con 0x7f2a381037c0 2024-09-12T02:16:58.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.863+0000 7f2a277fe640 1 -- 172.21.15.3:0/468777341 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2a3009c200 con 0x7f2a381037c0 2024-09-12T02:16:58.863 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-09-12T02:16:58.863 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQAaT+JmqLZnMxAA5B8ngF4DZHZ5P1o0l44IEg== 2024-09-12T02:16:58.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 msgr2=0x7f2a200a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 0x7f2a200a5100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a280058c0 tx=0x7f2a2800a4c0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 msgr2=0x7f2a381a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000eaa0 tx=0x7f2a3000ef60 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 shutdown_connections 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2a200a2c40 0x7f2a200a5100 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2a381037c0 0x7f2a381a16e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a38102e00 0x7f2a3819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 --2- 172.21.15.3:0/468777341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a38101c00 0x7f2a3819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.866+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 >> 172.21.15.3:0/468777341 conn(0x7f2a380fd390 msgr2=0x7f2a38074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.867+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 shutdown_connections 2024-09-12T02:16:58.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:16:58.867+0000 7f2a3cd6f640 1 -- 172.21.15.3:0/468777341 wait complete. 2024-09-12T02:16:59.666 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:16:59.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T02:16:59.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T02:16:59.706 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T02:16:59.706 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T02:16:59.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T02:17:00.005 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:00.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:59 smithi026 ceph-mon[35496]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-12T02:17:00.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:59 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:17:00.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:16:59 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:17:00.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:59 smithi003 ceph-mon[31476]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-12T02:17:00.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:59 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:17:00.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:16:59 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:17:00.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:59 smithi092 ceph-mon[36089]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-09-12T02:17:00.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:59 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:17:00.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:16:59 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/468777341' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 -- 172.21.15.3:0/3336488819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c074eb0 msgr2=0x7fc45c112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 --2- 172.21.15.3:0/3336488819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c074eb0 0x7fc45c112650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc450009900 tx=0x7fc45002a480 comp rx=0 tx=0).stop 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 -- 172.21.15.3:0/3336488819 shutdown_connections 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 --2- 172.21.15.3:0/3336488819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c112b90 0x7fc45c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 --2- 172.21.15.3:0/3336488819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c074eb0 0x7fc45c112650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 --2- 172.21.15.3:0/3336488819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074590 0x7fc45c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 -- 172.21.15.3:0/3336488819 >> 172.21.15.3:0/3336488819 conn(0x7fc45c0fd8c0 msgr2=0x7fc45c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 -- 172.21.15.3:0/3336488819 shutdown_connections 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.496+0000 7fc461313640 1 -- 172.21.15.3:0/3336488819 wait complete. 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 Processor -- start 2024-09-12T02:17:00.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 -- start start 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c074590 0x7fc45c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074eb0 0x7fc45c1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c118540 con 0x7fc45c112b90 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1183c0 con 0x7fc45c074590 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc461313640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1186c0 con 0x7fc45c074eb0 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc45b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc45b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56484/0 (socket says 172.21.15.3:56484) 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc45b7fe640 1 -- 172.21.15.3:0/636685471 learned_addr learned my addr 172.21.15.3:0/636685471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc45a7fc640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074eb0 0x7fc45c1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.497+0000 7fc45affd640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c074590 0x7fc45c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 -- 172.21.15.3:0/636685471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074eb0 msgr2=0x7fc45c1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:00.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074eb0 0x7fc45c1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 -- 172.21.15.3:0/636685471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c074590 msgr2=0x7fc45c1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c074590 0x7fc45c1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 -- 172.21.15.3:0/636685471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c19f000 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc45b7fe640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc45400cad0 tx=0x7fc45400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400ae20 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc454004030 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c19f2f0 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45400f7c0 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.498+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c1ab5e0 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.499+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c075cd0 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.500+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc4540041e0 con 0x7fc45c112b90 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.501+0000 7fc443fff640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 0x7fc4440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:00.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.501+0000 7fc45affd640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 0x7fc4440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.501+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc4540cf0b0 con 0x7fc45c112b90 2024-09-12T02:17:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.501+0000 7fc45affd640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 0x7fc4440a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc45c19fa10 tx=0x7fc44c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:00.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.505+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc454014070 con 0x7fc45c112b90 2024-09-12T02:17:00.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.726+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc45c074970 con 0x7fc45c112b90 2024-09-12T02:17:00.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.730+0000 7fc443fff640 1 -- 172.21.15.3:0/636685471 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263190 (secure 0 0 0) 0x7fc45c074970 con 0x7fc45c112b90 2024-09-12T02:17:00.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 msgr2=0x7fc4440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 0x7fc4440a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc45c19fa10 tx=0x7fc44c00b390 comp rx=0 tx=0).stop 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 msgr2=0x7fc45c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc45400cad0 tx=0x7fc45400cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 shutdown_connections 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc4440a2c90 0x7fc4440a5150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.734+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc45c112b90 0x7fc45c19e8d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.735+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc45c074eb0 0x7fc45c1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.735+0000 7fc461313640 1 --2- 172.21.15.3:0/636685471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc45c074590 0x7fc45c1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.735+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 >> 172.21.15.3:0/636685471 conn(0x7fc45c0fd8c0 msgr2=0x7fc45c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.735+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 shutdown_connections 2024-09-12T02:17:00.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:00.735+0000 7fc461313640 1 -- 172.21.15.3:0/636685471 wait complete. 2024-09-12T02:17:01.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:00 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/636685471' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:17:01.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:00 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/636685471' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:17:01.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:00 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/636685471' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:17:01.597 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2546806763},{"type":"v1","addr":"172.21.15.3:6801","nonce":2546806763}]},"active_addr":"172.21.15.3:6801/2546806763","active_change":"2024-09-12T02:09:36.346976+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3089375163}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":798360927}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2150548678}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3496966164}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":792586218}]}]} 2024-09-12T02:17:01.604 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T02:17:01.604 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T02:17:01.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:17:01.869 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:01.987 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:01 smithi003 ceph-mon[31476]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T02:17:02.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:01 smithi026 ceph-mon[35496]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T02:17:02.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:01 smithi092 ceph-mon[36089]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T02:17:02.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 -- 172.21.15.3:0/80896212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d5810c170 msgr2=0x7f6d5810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 --2- 172.21.15.3:0/80896212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d5810c170 0x7f6d5810e610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009ca0 tx=0x7f6d5002a950 comp rx=0 tx=0).stop 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 -- 172.21.15.3:0/80896212 shutdown_connections 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 --2- 172.21.15.3:0/80896212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d5810c170 0x7f6d5810e610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 --2- 172.21.15.3:0/80896212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d58103660 0x7f6d5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 --2- 172.21.15.3:0/80896212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 0x7f6d58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 -- 172.21.15.3:0/80896212 >> 172.21.15.3:0/80896212 conn(0x7f6d580fc7a0 msgr2=0x7f6d580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 -- 172.21.15.3:0/80896212 shutdown_connections 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.288+0000 7f6d5f469640 1 -- 172.21.15.3:0/80896212 wait complete. 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 Processor -- start 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 -- start start 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 0x7f6d58198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5810c170 0x7f6d5819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58111f70 con 0x7f6d58103660 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58111df0 con 0x7f6d58102d40 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5f469640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581120f0 con 0x7f6d5810c170 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5ca04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5ca04640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56498/0 (socket says 172.21.15.3:56498) 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.289+0000 7f6d5ca04640 1 -- 172.21.15.3:0/1094676627 learned_addr learned my addr 172.21.15.3:0/1094676627 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5da06640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5810c170 0x7f6d5819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5d205640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 0x7f6d58198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 -- 172.21.15.3:0/1094676627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5810c170 msgr2=0x7f6d5819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5810c170 0x7f6d5819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 -- 172.21.15.3:0/1094676627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 msgr2=0x7f6d58198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 0x7f6d58198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 -- 172.21.15.3:0/1094676627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5819d340 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5ca04640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c970 tx=0x7f6d4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48007bb0 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d48007d50 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5819d630 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800f450 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.290+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581a4f10 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.291+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d58103120 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.292+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6d48004f40 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.293+0000 7f6d4e7fc640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 0x7f6d440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.293+0000 7f6d5d205640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 0x7f6d440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.293+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6d480cf270 con 0x7f6d58103660 2024-09-12T02:17:02.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.294+0000 7f6d5d205640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 0x7f6d440a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d54005e10 tx=0x7f6d54005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:02.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.296+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d48014030 con 0x7f6d58103660 2024-09-12T02:17:02.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.475+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d581a5200 con 0x7f6d58103660 2024-09-12T02:17:02.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.477+0000 7f6d4e7fc640 1 -- 172.21.15.3:0/1094676627 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f6d4809c0e0 con 0x7f6d58103660 2024-09-12T02:17:02.477 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:02.477 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 msgr2=0x7f6d440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 0x7f6d440a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d54005e10 tx=0x7f6d54005da0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 msgr2=0x7f6d581988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800c970 tx=0x7f6d4800ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 shutdown_connections 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6d440a2c90 0x7f6d440a5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d5810c170 0x7f6d5819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6d58103660 0x7f6d581988d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 --2- 172.21.15.3:0/1094676627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d58102d40 0x7f6d58198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 >> 172.21.15.3:0/1094676627 conn(0x7f6d580fc7a0 msgr2=0x7f6d580fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 shutdown_connections 2024-09-12T02:17:02.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:02.479+0000 7f6d5f469640 1 -- 172.21.15.3:0/1094676627 wait complete. 2024-09-12T02:17:03.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:02 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1094676627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:03.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:02 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1094676627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:03.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:02 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1094676627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:03.342 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T02:17:03.342 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:17:03.606 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:04.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:03 smithi003 ceph-mon[31476]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 -- 172.21.15.3:0/4245838261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 msgr2=0x7fe7101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 --2- 172.21.15.3:0/4245838261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7101021c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe700009900 tx=0x7fe70002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 -- 172.21.15.3:0/4245838261 shutdown_connections 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 --2- 172.21.15.3:0/4245838261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710106f10 0x7fe71010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 --2- 172.21.15.3:0/4245838261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710102790 0x7fe7101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 --2- 172.21.15.3:0/4245838261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7101021c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 -- 172.21.15.3:0/4245838261 >> 172.21.15.3:0/4245838261 conn(0x7fe7100fd8c0 msgr2=0x7fe7100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 -- 172.21.15.3:0/4245838261 shutdown_connections 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.109+0000 7fe7158b9640 1 -- 172.21.15.3:0/4245838261 wait complete. 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 Processor -- start 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 -- start start 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710102790 0x7fe710074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 0x7fe710074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe710112010 con 0x7fe710101de0 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe710111e90 con 0x7fe710106f10 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe7158b9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe710112190 con 0x7fe710102790 2024-09-12T02:17:04.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe70effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe70effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56512/0 (socket says 172.21.15.3:56512) 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe70effd640 1 -- 172.21.15.3:0/361815119 learned_addr learned my addr 172.21.15.3:0/361815119 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.110+0000 7fe70e7fc640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710102790 0x7fe710074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70f7fe640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 0x7fe710074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 -- 172.21.15.3:0/361815119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710102790 msgr2=0x7fe710074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710102790 0x7fe710074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 -- 172.21.15.3:0/361815119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 msgr2=0x7fe710074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 0x7fe710074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 -- 172.21.15.3:0/361815119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7101a6a70 con 0x7fe710101de0 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70f7fe640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 0x7fe710074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe70effd640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe700005ab0 tx=0x7fe7000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70003f030 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7101a6d00 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7101a71d0 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe700003ce0 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.111+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700049060 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.112+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc005290 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.113+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe700042030 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.114+0000 7fe7148b7640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 0x7fe6f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.114+0000 7fe70e7fc640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 0x7fe6f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.114+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe7000ef910 con 0x7fe710101de0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.114+0000 7fe70e7fc640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 0x7fe6f80a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe710075f80 tx=0x7fe704009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:04.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.117+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7000bc700 con 0x7fe710101de0 2024-09-12T02:17:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:03 smithi026 ceph-mon[35496]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-12T02:17:04.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:03 smithi092 ceph-mon[36089]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-12T02:17:04.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.295+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6fc005850 con 0x7fe710101de0 2024-09-12T02:17:04.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.297+0000 7fe7148b7640 1 -- 172.21.15.3:0/361815119 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7fe7000dc0c0 con 0x7fe710101de0 2024-09-12T02:17:04.297 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:04.297 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:17:04.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 msgr2=0x7fe6f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 0x7fe6f80a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe710075f80 tx=0x7fe704009110 comp rx=0 tx=0).stop 2024-09-12T02:17:04.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 msgr2=0x7fe7100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe700005ab0 tx=0x7fe7000042f0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 shutdown_connections 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6f80a2c10 0x7fe6f80a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710106f10 0x7fe710074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe710102790 0x7fe710074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 --2- 172.21.15.3:0/361815119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe710101de0 0x7fe7100744f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 >> 172.21.15.3:0/361815119 conn(0x7fe7100fd8c0 msgr2=0x7fe7100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 shutdown_connections 2024-09-12T02:17:04.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:04.299+0000 7fe7158b9640 1 -- 172.21.15.3:0/361815119 wait complete. 2024-09-12T02:17:05.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:04 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/361815119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:05.067 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T02:17:05.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T02:17:05.068 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T02:17:05.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T02:17:05.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T02:17:05.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T02:17:05.070 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T02:17:05.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T02:17:05.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T02:17:05.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T02:17:05.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T02:17:05.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T02:17:05.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:04 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/361815119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:05.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:04 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/361815119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:05.881 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.881 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.883 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.886 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.887 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.891 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.894 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.897 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.901 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:05.908 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:06.053 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:05 smithi003 ceph-mon[31476]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:17:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:05 smithi026 ceph-mon[35496]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:17:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:05 smithi092 ceph-mon[36089]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:17:06.393 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:06.397 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 -- 172.21.15.3:0/2741721552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e01036e0 msgr2=0x7f06e0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2741721552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e01036e0 0x7f06e0107b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009930 tx=0x7f06c802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 -- 172.21.15.3:0/2741721552 shutdown_connections 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2741721552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06e01082a0 0x7f06e010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2741721552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e01036e0 0x7f06e0107b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2741721552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e0102d30 0x7f06e0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 -- 172.21.15.3:0/2741721552 >> 172.21.15.3:0/2741721552 conn(0x7f06e00fc7d0 msgr2=0x7f06e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:06.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 -- 172.21.15.3:0/2741721552 shutdown_connections 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.881+0000 7f06e4c12640 1 -- 172.21.15.3:0/2741721552 wait complete. 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 Processor -- start 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 -- start start 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 0x7f06e0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06e01036e0 0x7f06e0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e01082a0 0x7f06e019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01120a0 con 0x7f06e0102d30 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0111f20 con 0x7f06e01082a0 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06e4c12640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0112220 con 0x7f06e01036e0 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 0x7f06e0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 0x7f06e0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56542/0 (socket says 172.21.15.3:56542) 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 -- 172.21.15.3:0/2436093893 learned_addr learned my addr 172.21.15.3:0/2436093893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:06.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06d7fff640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06e01036e0 0x7f06e0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06ded9d640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e01082a0 0x7f06e019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 -- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06e01036e0 msgr2=0x7f06e0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06e01036e0 0x7f06e0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 -- 172.21.15.3:0/2436093893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e01082a0 msgr2=0x7f06e019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e01082a0 0x7f06e019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.882+0000 7f06de59c640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e019d540 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06de59c640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 0x7f06e0198590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d950 tx=0x7f06d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0017070 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e019d830 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0009c30 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d000f6b0 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.883+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01a5110 con 0x7f06e0102d30 2024-09-12T02:17:06.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.884+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f06e0062980 con 0x7f06e0102d30 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.885+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f06d000f850 con 0x7f06e0102d30 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d77fe640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d7fff640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f06d00cf310 con 0x7f06e0102d30 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d77fe640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] conn(0x7f06cc0ac840 0x7f06cc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06d0001d30 con 0x7f06cc0ac840 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.886+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f06d00cf760 con 0x7f06e0102d30 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.887+0000 7f06ded9d640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] conn(0x7f06cc0ac840 0x7f06cc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.887+0000 7f06d7fff640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005940 tx=0x7f06c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:06.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.887+0000 7f06ded9d640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] conn(0x7f06cc0ac840 0x7f06cc0aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:06.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.888+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== osd.5 v2:172.21.15.92:6808/2340798273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d0001d30 con 0x7f06cc0ac840 2024-09-12T02:17:06.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.902+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 --> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f06e0062e00 con 0x7f06cc0ac840 2024-09-12T02:17:06.902 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.902+0000 7f06d77fe640 1 -- 172.21.15.3:0/2436093893 <== osd.5 v2:172.21.15.92:6808/2340798273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f06e0062e00 con 0x7f06cc0ac840 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] conn(0x7f06cc0ac840 msgr2=0x7f06cc0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] conn(0x7f06cc0ac840 0x7f06cc0aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06cc0a2c10 msgr2=0x7f06cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06cc0a2c10 0x7f06cc0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005940 tx=0x7f06c80058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 msgr2=0x7f06e0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 --2- 172.21.15.3:0/2436093893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f06e0102d30 0x7f06e0198590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06d000d950 tx=0x7f06d000de10 comp rx=0 tx=0).stop 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06ded9d640 1 -- 172.21.15.3:0/2436093893 reap_dead start 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 shutdown_connections 2024-09-12T02:17:06.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 >> 172.21.15.3:0/2436093893 conn(0x7f06e00fc7d0 msgr2=0x7f06e0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 shutdown_connections 2024-09-12T02:17:06.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:06.903+0000 7f06e4c12640 1 -- 172.21.15.3:0/2436093893 wait complete. 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 -- 172.21.15.3:0/697159260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c0102d40 msgr2=0x7f71c0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 --2- 172.21.15.3:0/697159260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c0102d40 0x7f71c0103120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02a480 comp rx=0 tx=0).stop 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 -- 172.21.15.3:0/697159260 shutdown_connections 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 --2- 172.21.15.3:0/697159260 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c010c170 0x7f71c010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 --2- 172.21.15.3:0/697159260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71c0103660 0x7f71c010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 --2- 172.21.15.3:0/697159260 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c0102d40 0x7f71c0103120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 -- 172.21.15.3:0/697159260 >> 172.21.15.3:0/697159260 conn(0x7f71c00fc7a0 msgr2=0x7f71c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 -- 172.21.15.3:0/697159260 shutdown_connections 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 -- 172.21.15.3:0/697159260 wait complete. 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.917+0000 7f71c7791640 1 Processor -- start 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 -- start start 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c0102d40 0x7f71c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71c0103660 0x7f71c019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 0x7f71c019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111f70 con 0x7f71c010c170 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0111df0 con 0x7f71c0103660 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c7791640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01120f0 con 0x7f71c0102d40 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 0x7f71c019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:07.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 0x7f71c019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56560/0 (socket says 172.21.15.3:56560) 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 -- 172.21.15.3:0/130004643 learned_addr learned my addr 172.21.15.3:0/130004643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c552d640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c0102d40 0x7f71c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c4d2c640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71c0103660 0x7f71c019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 -- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c0102d40 msgr2=0x7f71c019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71c0102d40 0x7f71c019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 -- 172.21.15.3:0/130004643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71c0103660 msgr2=0x7f71c019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71c0103660 0x7f71c019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019f620 con 0x7f71c010c170 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.918+0000 7f71c5d2e640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 0x7f71c019ef40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71b800cad0 tx=0x7f71b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.919+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800ae20 con 0x7f71c010c170 2024-09-12T02:17:07.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.919+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b8005020 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.919+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019f910 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.919+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b800f6f0 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.919+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a71f0 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.920+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f71c0102500 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.921+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f71b80051d0 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.921+0000 7f71b67fc640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f71ac0a2c10 0x7f71ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.921+0000 7f71c552d640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f71ac0a2c10 0x7f71ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.921+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f71b80cf5e0 con 0x7f71c010c170 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71b67fc640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] conn(0x7f71ac0ac840 0x7f71ac0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71b8012ec0 con 0x7f71ac0ac840 2024-09-12T02:17:07.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f71b80cfa30 con 0x7f71c010c170 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71c4d2c640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] conn(0x7f71ac0ac840 0x7f71ac0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71c552d640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f71ac0a2c10 0x7f71ac0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009230 tx=0x7f71bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.922+0000 7f71c4d2c640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] conn(0x7f71ac0ac840 0x7f71ac0aece0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.923+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== osd.11 v2:172.21.15.92:6824/3259822775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71b8012ec0 con 0x7f71ac0ac840 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.937+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 --> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71c006b070 con 0x7f71ac0ac840 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71b67fc640 1 -- 172.21.15.3:0/130004643 <== osd.11 v2:172.21.15.92:6824/3259822775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71c006b070 con 0x7f71ac0ac840 2024-09-12T02:17:07.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] conn(0x7f71ac0ac840 msgr2=0x7f71ac0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] conn(0x7f71ac0ac840 0x7f71ac0aece0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f71ac0a2c10 msgr2=0x7f71ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f71ac0a2c10 0x7f71ac0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009230 tx=0x7f71bc0058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 msgr2=0x7f71c019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 --2- 172.21.15.3:0/130004643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f71c010c170 0x7f71c019ef40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71b800cad0 tx=0x7f71b800cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c5d2e640 1 -- 172.21.15.3:0/130004643 reap_dead start 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 shutdown_connections 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 >> 172.21.15.3:0/130004643 conn(0x7f71c00fc7a0 msgr2=0x7f71c0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 shutdown_connections 2024-09-12T02:17:07.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:07.938+0000 7f71c7791640 1 -- 172.21.15.3:0/130004643 wait complete. 2024-09-12T02:17:08.075 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:07 smithi003 ceph-mon[31476]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:17:08.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:07 smithi026 ceph-mon[35496]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:17:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:07 smithi092 ceph-mon[36089]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:17:08.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.402+0000 7f1111c53640 1 -- 172.21.15.3:0/1462939232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101fc0 msgr2=0x7f110c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.402+0000 7f1111c53640 1 --2- 172.21.15.3:0/1462939232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101fc0 0x7f110c105710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1100009900 tx=0x7f110002a410 comp rx=0 tx=0).stop 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 -- 172.21.15.3:0/1462939232 shutdown_connections 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 --2- 172.21.15.3:0/1462939232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f110c105e40 0x7f110c10d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 --2- 172.21.15.3:0/1462939232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c101fc0 0x7f110c105710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 --2- 172.21.15.3:0/1462939232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f110c101610 0x7f110c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 -- 172.21.15.3:0/1462939232 >> 172.21.15.3:0/1462939232 conn(0x7f110c0fc5c0 msgr2=0x7f110c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 -- 172.21.15.3:0/1462939232 shutdown_connections 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 -- 172.21.15.3:0/1462939232 wait complete. 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.403+0000 7f1111c53640 1 Processor -- start 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 -- start start 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f110c101610 0x7f110c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:08.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f110c101fc0 0x7f110c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 0x7f110c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f110c110ea0 con 0x7f110c105e40 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f110c110d20 con 0x7f110c101610 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f1111c53640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f110c111020 con 0x7f110c101fc0 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 0x7f110c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:08.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 0x7f110c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56572/0 (socket says 172.21.15.3:56572) 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 -- 172.21.15.3:0/983283828 learned_addr learned my addr 172.21.15.3:0/983283828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110affd640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f110c101fc0 0x7f110c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110b7fe640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f110c101610 0x7f110c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 -- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f110c101fc0 msgr2=0x7f110c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f110c101fc0 0x7f110c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 -- 172.21.15.3:0/983283828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f110c101610 msgr2=0x7f110c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f110c101610 0x7f110c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.404+0000 7f110bfff640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c19d410 con 0x7f110c105e40 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f110bfff640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 0x7f110c19cd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f110400cad0 tx=0x7f110400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:08.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110400ae20 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f1111c53640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c19d700 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1104005020 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110400f6f0 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.405+0000 7f1111c53640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c0fc9a0 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.406+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f110c062980 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.407+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f11040051d0 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f1108ff9640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10f40a2c10 0x7f10f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f110b7fe640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10f40a2c10 0x7f10f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f11040cf580 con 0x7f110c105e40 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f1108ff9640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] conn(0x7f10f40ac840 0x7f10f40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10f40af3b0 con 0x7f10f40ac840 2024-09-12T02:17:08.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f11040cf930 con 0x7f110c105e40 2024-09-12T02:17:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.408+0000 7f110b7fe640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10f40a2c10 0x7f10f40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00bbe0 tx=0x7f10fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.409+0000 7f110affd640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] conn(0x7f10f40ac840 0x7f10f40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.409+0000 7f110affd640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] conn(0x7f10f40ac840 0x7f10f40aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:08.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.410+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== osd.2 v2:172.21.15.92:6800/3785909274 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10f40af3b0 con 0x7f10f40ac840 2024-09-12T02:17:08.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.424+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 --> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f110c062e00 con 0x7f10f40ac840 2024-09-12T02:17:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.425+0000 7f1108ff9640 1 -- 172.21.15.3:0/983283828 <== osd.2 v2:172.21.15.92:6800/3785909274 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f110c062e00 con 0x7f10f40ac840 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] conn(0x7f10f40ac840 msgr2=0x7f10f40aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] conn(0x7f10f40ac840 0x7f10f40aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10f40a2c10 msgr2=0x7f10f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f10f40a2c10 0x7f10f40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00bbe0 tx=0x7f10fc00b390 comp rx=0 tx=0).stop 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 msgr2=0x7f110c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:08.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 --2- 172.21.15.3:0/983283828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f110c105e40 0x7f110c19cd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f110400cad0 tx=0x7f110400cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f110bfff640 1 -- 172.21.15.3:0/983283828 reap_dead start 2024-09-12T02:17:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 shutdown_connections 2024-09-12T02:17:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 >> 172.21.15.3:0/983283828 conn(0x7f110c0fc5c0 msgr2=0x7f110c104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 shutdown_connections 2024-09-12T02:17:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:08.426+0000 7f10f27fc640 1 -- 172.21.15.3:0/983283828 wait complete. 2024-09-12T02:17:09.262 INFO:teuthology.orchestra.run.smithi003.stdout:73014444099 2024-09-12T02:17:09.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T02:17:09.976 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:09 smithi003 ceph-mon[31476]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-12T02:17:10.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 -- 172.21.15.3:0/3665473046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 msgr2=0x7f575010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 --2- 172.21.15.3:0/3665473046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575010ea90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5748009ca0 tx=0x7f574802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 -- 172.21.15.3:0/3665473046 shutdown_connections 2024-09-12T02:17:10.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 --2- 172.21.15.3:0/3665473046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575010ea90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 --2- 172.21.15.3:0/3665473046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5750102790 0x7f57501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 --2- 172.21.15.3:0/3665473046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 0x7f57501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 -- 172.21.15.3:0/3665473046 >> 172.21.15.3:0/3665473046 conn(0x7f57500fd8c0 msgr2=0x7f57500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 -- 172.21.15.3:0/3665473046 shutdown_connections 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.092+0000 7f57586f2640 1 -- 172.21.15.3:0/3665473046 wait complete. 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 Processor -- start 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 -- start start 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 0x7f57501983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5750102790 0x7f5750198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5750111f70 con 0x7f5750106f10 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5750111df0 con 0x7f5750102790 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f57586f2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f57501120f0 con 0x7f5750101de0 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59330/0 (socket says 172.21.15.3:59330) 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 -- 172.21.15.3:0/1831322488 learned_addr learned my addr 172.21.15.3:0/1831322488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f575648e640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 0x7f57501983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5755c8d640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5750102790 0x7f5750198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 -- 172.21.15.3:0/1831322488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 msgr2=0x7f57501983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 0x7f57501983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 -- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5750102790 msgr2=0x7f5750198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5750102790 0x7f5750198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f5756c8f640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019d380 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.093+0000 7f575648e640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5750101de0 0x7f57501983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f5756c8f640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575019cca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5748031a60 tx=0x7f5748031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5748003c30 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575019d610 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5748003dd0 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57480047e0 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.094+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a4f50 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.095+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5750062980 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.096+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f574802f790 con 0x7f5750106f10 2024-09-12T02:17:10.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f57477fe640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f573c0a2c10 0x7f573c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f575648e640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f573c0a2c10 0x7f573c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f57480f0760 con 0x7f5750106f10 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f57477fe640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] conn(0x7f573c0ac840 0x7f573c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f573c0af3b0 con 0x7f573c0ac840 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f57480f0bb0 con 0x7f5750106f10 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f575648e640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f573c0a2c10 0x7f573c0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f574c0058c0 tx=0x7f574c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.097+0000 7f5755c8d640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] conn(0x7f573c0ac840 0x7f573c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.098+0000 7f5755c8d640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] conn(0x7f573c0ac840 0x7f573c0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.099+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== osd.4 v2:172.21.15.26:6808/525329948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f573c0af3b0 con 0x7f573c0ac840 2024-09-12T02:17:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.112+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 --> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5750062e00 con 0x7f573c0ac840 2024-09-12T02:17:10.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57477fe640 1 -- 172.21.15.3:0/1831322488 <== osd.4 v2:172.21.15.26:6808/525329948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5750062e00 con 0x7f573c0ac840 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] conn(0x7f573c0ac840 msgr2=0x7f573c0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] conn(0x7f573c0ac840 0x7f573c0aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f573c0a2c10 msgr2=0x7f573c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f573c0a2c10 0x7f573c0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f574c0058c0 tx=0x7f574c00a250 comp rx=0 tx=0).stop 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 msgr2=0x7f575019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 --2- 172.21.15.3:0/1831322488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5750106f10 0x7f575019cca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5748031a60 tx=0x7f5748031a90 comp rx=0 tx=0).stop 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f5756c8f640 1 -- 172.21.15.3:0/1831322488 reap_dead start 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 shutdown_connections 2024-09-12T02:17:10.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 >> 172.21.15.3:0/1831322488 conn(0x7f57500fd8c0 msgr2=0x7f57500fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 shutdown_connections 2024-09-12T02:17:10.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.113+0000 7f57586f2640 1 -- 172.21.15.3:0/1831322488 wait complete. 2024-09-12T02:17:10.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:09 smithi026 ceph-mon[35496]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-12T02:17:10.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:09 smithi092 ceph-mon[36089]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 -- 172.21.15.3:0/480900188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 msgr2=0x7fb21c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 --2- 172.21.15.3:0/480900188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c10ea90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb214009ca0 tx=0x7fb21402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 -- 172.21.15.3:0/480900188 shutdown_connections 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 --2- 172.21.15.3:0/480900188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c10ea90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 --2- 172.21.15.3:0/480900188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb21c102790 0x7fb21c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 --2- 172.21.15.3:0/480900188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb21c101de0 0x7fb21c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 -- 172.21.15.3:0/480900188 >> 172.21.15.3:0/480900188 conn(0x7fb21c0fd8c0 msgr2=0x7fb21c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 -- 172.21.15.3:0/480900188 shutdown_connections 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.522+0000 7fb22211a640 1 -- 172.21.15.3:0/480900188 wait complete. 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 Processor -- start 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 -- start start 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb21c101de0 0x7fb21c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb21c102790 0x7fb21c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c111f70 con 0x7fb21c106f10 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c111df0 con 0x7fb21c101de0 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb22211a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1120f0 con 0x7fb21c102790 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb21bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb21bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59352/0 (socket says 172.21.15.3:59352) 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb21bfff640 1 -- 172.21.15.3:0/3663131922 learned_addr learned my addr 172.21.15.3:0/3663131922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:10.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb213fff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb21c102790 0x7fb21c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.523+0000 7fb21b7fe640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb21c101de0 0x7fb21c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 -- 172.21.15.3:0/3663131922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb21c102790 msgr2=0x7fb21c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb21c102790 0x7fb21c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 -- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb21c101de0 msgr2=0x7fb21c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb21c101de0 0x7fb21c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c19d380 con 0x7fb21c106f10 2024-09-12T02:17:10.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb21bfff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c19cca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb214031a20 tx=0x7fb214031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214003c30 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb214003dd0 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c19d610 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21402fcf0 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.524+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1a4f50 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.525+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb21c062980 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.526+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb214004050 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb2197fa640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2080a2c90 0x7fb2080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2140f05b0 con 0x7fb21c106f10 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb21b7fe640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2080a2c90 0x7fb2080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb2197fa640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] conn(0x7fb2080ac8c0 0x7fb2080aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2080af430 con 0x7fb2080ac8c0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fb2140f0a00 con 0x7fb21c106f10 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.527+0000 7fb21b7fe640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2080a2c90 0x7fb2080a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005fd0 tx=0x7fb20c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.528+0000 7fb213fff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] conn(0x7fb2080ac8c0 0x7fb2080aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.528+0000 7fb213fff640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] conn(0x7fb2080ac8c0 0x7fb2080aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.529+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== osd.7 v2:172.21.15.26:6816/500611310 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2080af430 con 0x7fb2080ac8c0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 --> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb21c062d80 con 0x7fb2080ac8c0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb2197fa640 1 -- 172.21.15.3:0/3663131922 <== osd.7 v2:172.21.15.26:6816/500611310 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb21c062d80 con 0x7fb2080ac8c0 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] conn(0x7fb2080ac8c0 msgr2=0x7fb2080aed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] conn(0x7fb2080ac8c0 0x7fb2080aed60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2080a2c90 msgr2=0x7fb2080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2080a2c90 0x7fb2080a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005fd0 tx=0x7fb20c00b390 comp rx=0 tx=0).stop 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 msgr2=0x7fb21c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 --2- 172.21.15.3:0/3663131922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb21c106f10 0x7fb21c19cca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb214031a20 tx=0x7fb214031a50 comp rx=0 tx=0).stop 2024-09-12T02:17:10.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb21bfff640 1 -- 172.21.15.3:0/3663131922 reap_dead start 2024-09-12T02:17:10.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.543+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 shutdown_connections 2024-09-12T02:17:10.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.544+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 >> 172.21.15.3:0/3663131922 conn(0x7fb21c0fd8c0 msgr2=0x7fb21c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.544+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 shutdown_connections 2024-09-12T02:17:10.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.544+0000 7fb22211a640 1 -- 172.21.15.3:0/3663131922 wait complete. 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3264984390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8106a80 msgr2=0x7f32f8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3264984390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8106a80 0x7f32f8106e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009930 tx=0x7f32e802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3264984390 shutdown_connections 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3264984390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8101420 0x7f32f810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3264984390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8100a80 0x7f32f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3264984390 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8106a80 0x7f32f8106e60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3264984390 >> 172.21.15.3:0/3264984390 conn(0x7f32f80fc7a0 msgr2=0x7f32f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3264984390 shutdown_connections 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3264984390 wait complete. 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.975+0000 7f32fe1f5640 1 Processor -- start 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 -- start start 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8100a80 0x7f32f81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 0x7f32f8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8106a80 0x7f32f81a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8110e80 con 0x7f32f8101420 2024-09-12T02:17:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8110d00 con 0x7f32f8100a80 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32fe1f5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f32f8111000 con 0x7f32f8106a80 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 0x7f32f8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 0x7f32f8199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59370/0 (socket says 172.21.15.3:59370) 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 -- 172.21.15.3:0/3604479937 learned_addr learned my addr 172.21.15.3:0/3604479937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f77fe640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8100a80 0x7f32f81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f7fff640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8106a80 0x7f32f81a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 -- 172.21.15.3:0/3604479937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8106a80 msgr2=0x7f32f81a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8106a80 0x7f32f81a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 -- 172.21.15.3:0/3604479937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8100a80 msgr2=0x7f32f81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32f8100a80 0x7f32f81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f81a4890 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f7fff640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f8106a80 0x7f32f81a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.976+0000 7f32f6ffd640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 0x7f32f8199810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00d950 tx=0x7f32ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec017070 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f81a4a60 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec009c30 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00f6b0 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a4f50 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.977+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32f8062900 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.979+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f32ec0109d0 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f4ff9640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32e00a2c10 0x7f32e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f32ec0cf1c0 con 0x7f32f8101420 2024-09-12T02:17:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f77fe640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32e00a2c10 0x7f32e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f4ff9640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] conn(0x7f32e00ac840 0x7f32e00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32e00af3b0 con 0x7f32e00ac840 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f32ec0cf610 con 0x7f32f8101420 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f7fff640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] conn(0x7f32e00ac840 0x7f32e00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f77fe640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32e00a2c10 0x7f32e00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.980+0000 7f32f7fff640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] conn(0x7f32e00ac840 0x7f32e00aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.981+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== osd.0 v2:172.21.15.3:6802/46072928 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32e00af3b0 con 0x7f32e00ac840 2024-09-12T02:17:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.995+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 --> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32f8062d80 con 0x7f32e00ac840 2024-09-12T02:17:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.995+0000 7f32f4ff9640 1 -- 172.21.15.3:0/3604479937 <== osd.0 v2:172.21.15.3:6802/46072928 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32f8062d80 con 0x7f32e00ac840 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.995+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] conn(0x7f32e00ac840 msgr2=0x7f32e00aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.995+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] conn(0x7f32e00ac840 0x7f32e00aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.995+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32e00a2c10 msgr2=0x7f32e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f32e00a2c10 0x7f32e00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e80058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 msgr2=0x7f32f8199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 --2- 172.21.15.3:0/3604479937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f32f8101420 0x7f32f8199810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00d950 tx=0x7f32ec00de10 comp rx=0 tx=0).stop 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32f7fff640 1 -- 172.21.15.3:0/3604479937 reap_dead start 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 shutdown_connections 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 >> 172.21.15.3:0/3604479937 conn(0x7f32f80fc7a0 msgr2=0x7f32f80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 shutdown_connections 2024-09-12T02:17:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:10.996+0000 7f32fe1f5640 1 -- 172.21.15.3:0/3604479937 wait complete. 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 -- 172.21.15.3:0/232765680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c103740 msgr2=0x7ffa8c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 --2- 172.21.15.3:0/232765680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c103740 0x7ffa8c107b80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009930 tx=0x7ffa7402a420 comp rx=0 tx=0).stop 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 -- 172.21.15.3:0/232765680 shutdown_connections 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 --2- 172.21.15.3:0/232765680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8c108290 0x7ffa8c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 --2- 172.21.15.3:0/232765680 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c103740 0x7ffa8c107b80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 --2- 172.21.15.3:0/232765680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 0x7ffa8c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 -- 172.21.15.3:0/232765680 >> 172.21.15.3:0/232765680 conn(0x7ffa8c0fc7d0 msgr2=0x7ffa8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:11.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 -- 172.21.15.3:0/232765680 shutdown_connections 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.193+0000 7ffa928c8640 1 -- 172.21.15.3:0/232765680 wait complete. 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 Processor -- start 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 -- start start 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 0x7ffa8c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8c103740 0x7ffa8c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 0x7ffa8c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1120b0 con 0x7ffa8c108290 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c111f30 con 0x7ffa8c102d90 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa928c8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112230 con 0x7ffa8c103740 2024-09-12T02:17:11.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa90e65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 0x7ffa8c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 0x7ffa8c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa90e65640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 0x7ffa8c19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59386/0 (socket says 172.21.15.3:59386) 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.194+0000 7ffa90e65640 1 -- 172.21.15.3:0/2263171797 learned_addr learned my addr 172.21.15.3:0/2263171797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa83fff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8c103740 0x7ffa8c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 -- 172.21.15.3:0/2263171797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8c103740 msgr2=0x7ffa8c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa8c103740 0x7ffa8c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 -- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 msgr2=0x7ffa8c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 0x7ffa8c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c19d5c0 con 0x7ffa8c102d90 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa90e65640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa8c108290 0x7ffa8c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa8bfff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 0x7ffa8c198610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e9f0 tx=0x7ffa7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c00ccc0 con 0x7ffa8c102d90 2024-09-12T02:17:11.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c19d8b0 con 0x7ffa8c102d90 2024-09-12T02:17:11.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c010910 con 0x7ffa8c102d90 2024-09-12T02:17:11.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c018960 con 0x7ffa8c102d90 2024-09-12T02:17:11.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.195+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a5190 con 0x7ffa8c102d90 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.197+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa8c062980 con 0x7ffa8c102d90 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.198+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffa7c010ac0 con 0x7ffa8c102d90 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.198+0000 7ffa89ffb640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa780a2c10 0x7ffa780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa83fff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa780a2c10 0x7ffa780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ffa7c0cfb20 con 0x7ffa8c102d90 2024-09-12T02:17:11.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa89ffb640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] conn(0x7ffa780ac840 0x7ffa780aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa780af3b0 con 0x7ffa780ac840 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c0cfed0 con 0x7ffa8c102d90 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa90e65640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] conn(0x7ffa780ac840 0x7ffa780aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa83fff640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa780a2c10 0x7ffa780a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa74006010 tx=0x7ffa740314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.199+0000 7ffa90e65640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] conn(0x7ffa780ac840 0x7ffa780aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:11.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.200+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== osd.6 v2:172.21.15.3:6818/342811658 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa780af3b0 con 0x7ffa780ac840 2024-09-12T02:17:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.214+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 --> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa8c062e00 con 0x7ffa780ac840 2024-09-12T02:17:11.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.214+0000 7ffa89ffb640 1 -- 172.21.15.3:0/2263171797 <== osd.6 v2:172.21.15.3:6818/342811658 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa8c062e00 con 0x7ffa780ac840 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] conn(0x7ffa780ac840 msgr2=0x7ffa780aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] conn(0x7ffa780ac840 0x7ffa780aece0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa780a2c10 msgr2=0x7ffa780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ffa780a2c10 0x7ffa780a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa74006010 tx=0x7ffa740314b0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 msgr2=0x7ffa8c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:11.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 --2- 172.21.15.3:0/2263171797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8c102d90 0x7ffa8c198610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00e9f0 tx=0x7ffa7c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:17:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa90e65640 1 -- 172.21.15.3:0/2263171797 reap_dead start 2024-09-12T02:17:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 shutdown_connections 2024-09-12T02:17:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 >> 172.21.15.3:0/2263171797 conn(0x7ffa8c0fc7d0 msgr2=0x7ffa8c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 shutdown_connections 2024-09-12T02:17:11.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:11.215+0000 7ffa928c8640 1 -- 172.21.15.3:0/2263171797 wait complete. 2024-09-12T02:17:11.928 INFO:teuthology.orchestra.run.smithi003.stdout:210453397546 2024-09-12T02:17:11.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T02:17:12.113 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:11 smithi003 ceph-mon[31476]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:17:12.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:11 smithi026 ceph-mon[35496]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:17:12.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:11 smithi092 ceph-mon[36089]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:17:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 -- 172.21.15.3:0/2017300790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238102790 msgr2=0x7f52381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 --2- 172.21.15.3:0/2017300790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238102790 0x7f52381067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f522c009900 tx=0x7f522c02a410 comp rx=0 tx=0).stop 2024-09-12T02:17:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 -- 172.21.15.3:0/2017300790 shutdown_connections 2024-09-12T02:17:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 --2- 172.21.15.3:0/2017300790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5238106f10 0x7f523810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 --2- 172.21.15.3:0/2017300790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238102790 0x7f52381067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 --2- 172.21.15.3:0/2017300790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238101de0 0x7f52381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 -- 172.21.15.3:0/2017300790 >> 172.21.15.3:0/2017300790 conn(0x7f52380fd8c0 msgr2=0x7f52380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 -- 172.21.15.3:0/2017300790 shutdown_connections 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.951+0000 7f523e035640 1 -- 172.21.15.3:0/2017300790 wait complete. 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 Processor -- start 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 -- start start 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 0x7f5238198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238102790 0x7f5238198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5238106f10 0x7f523819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111f70 con 0x7f5238101de0 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111df0 con 0x7f5238102790 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f523e035640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f52381120f0 con 0x7f5238106f10 2024-09-12T02:17:12.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 0x7f5238198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 0x7f5238198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59414/0 (socket says 172.21.15.3:59414) 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 -- 172.21.15.3:0/124908886 learned_addr learned my addr 172.21.15.3:0/124908886 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f5236ffd640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238102790 0x7f5238198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 -- 172.21.15.3:0/124908886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5238106f10 msgr2=0x7f523819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5238106f10 0x7f523819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 -- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238102790 msgr2=0x7f5238198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.952+0000 7f52377fe640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5238102790 0x7f5238198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f52377fe640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523819d3e0 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f52377fe640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 0x7f5238198430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f522800d950 tx=0x7f522800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228017030 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523819d6d0 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5228004020 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52280047a0 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.953+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a4fb0 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.954+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5238062940 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.955+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f52280041d0 con 0x7f5238101de0 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5234ff9640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f52200a2c10 0x7f52200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:12.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5236ffd640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f52200a2c10 0x7f52200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f52280cf330 con 0x7f5238101de0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5234ff9640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] conn(0x7f52200ac840 0x7f52200aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52200af3b0 con 0x7f52200ac840 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f52280cf780 con 0x7f5238101de0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5236ffd640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f52200a2c10 0x7f52200a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f522c0096c0 tx=0x7f522c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.956+0000 7f5237fff640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] conn(0x7f52200ac840 0x7f52200aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.957+0000 7f5237fff640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] conn(0x7f52200ac840 0x7f52200aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.958+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== osd.10 v2:172.21.15.26:6824/3810836866 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52200af3b0 con 0x7f52200ac840 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 --> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5238062d70 con 0x7f52200ac840 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f5234ff9640 1 -- 172.21.15.3:0/124908886 <== osd.10 v2:172.21.15.26:6824/3810836866 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5238062d70 con 0x7f52200ac840 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] conn(0x7f52200ac840 msgr2=0x7f52200aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] conn(0x7f52200ac840 0x7f52200aece0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f52200a2c10 msgr2=0x7f52200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f52200a2c10 0x7f52200a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f522c0096c0 tx=0x7f522c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 msgr2=0x7f5238198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 --2- 172.21.15.3:0/124908886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5238101de0 0x7f5238198430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f522800d950 tx=0x7f522800de10 comp rx=0 tx=0).stop 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f5237fff640 1 -- 172.21.15.3:0/124908886 reap_dead start 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 shutdown_connections 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 >> 172.21.15.3:0/124908886 conn(0x7f52380fd8c0 msgr2=0x7f52380fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 shutdown_connections 2024-09-12T02:17:12.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.972+0000 7f523e035640 1 -- 172.21.15.3:0/124908886 wait complete. 2024-09-12T02:17:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 -- 172.21.15.3:0/2882587844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4100a80 msgr2=0x7f0de4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2882587844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4100a80 0x7f0de4100ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009930 tx=0x7f0dd802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 -- 172.21.15.3:0/2882587844 shutdown_connections 2024-09-12T02:17:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2882587844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4101420 0x7f0de410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2882587844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4100a80 0x7f0de4100ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2882587844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de4106a80 0x7f0de4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 -- 172.21.15.3:0/2882587844 >> 172.21.15.3:0/2882587844 conn(0x7f0de40fc7a0 msgr2=0x7f0de40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 -- 172.21.15.3:0/2882587844 shutdown_connections 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.635+0000 7f0de8c63640 1 -- 172.21.15.3:0/2882587844 wait complete. 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 Processor -- start 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 -- start start 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4100a80 0x7f0de41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 0x7f0de41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de4106a80 0x7f0de41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4110ef0 con 0x7f0de4101420 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4110d70 con 0x7f0de4100a80 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de8c63640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4111070 con 0x7f0de4106a80 2024-09-12T02:17:13.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 0x7f0de41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 0x7f0de41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59394/0 (socket says 172.21.15.3:59394) 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de1d9b640 1 -- 172.21.15.3:0/2812017394 learned_addr learned my addr 172.21.15.3:0/2812017394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.636+0000 7f0de259c640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4100a80 0x7f0de41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de2d9d640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de4106a80 0x7f0de41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 -- 172.21.15.3:0/2812017394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de4106a80 msgr2=0x7f0de41a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0de4106a80 0x7f0de41a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 -- 172.21.15.3:0/2812017394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4100a80 msgr2=0x7f0de41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4100a80 0x7f0de41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de41a4860 con 0x7f0de4101420 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de259c640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0de4100a80 0x7f0de41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de1d9b640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 0x7f0de41997e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0dd802a940 tx=0x7f0dd8003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:13.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd803f070 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de41a4a30 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd803b460 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8035030 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.637+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de41a4e80 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.638+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0de40fd750 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.639+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0dd8042070 con 0x7f0de4101420 2024-09-12T02:17:13.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0dd37fe640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0dcc0a2c10 0x7f0dcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0de259c640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0dcc0a2c10 0x7f0dcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0dd80eff60 con 0x7f0de4101420 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0dd37fe640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] conn(0x7f0dcc0ac840 0x7f0dcc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0dd800b0b0 con 0x7f0dcc0ac840 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0dd80f0310 con 0x7f0de4101420 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0de2d9d640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] conn(0x7f0dcc0ac840 0x7f0dcc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.640+0000 7f0de259c640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0dcc0a2c10 0x7f0dcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400abe0 tx=0x7f0dd400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.641+0000 7f0de2d9d640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] conn(0x7f0dcc0ac840 0x7f0dcc0aece0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.641+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== osd.9 v2:172.21.15.3:6826/3263587345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0dd800b0b0 con 0x7f0dcc0ac840 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.655+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 --> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0de41021c0 con 0x7f0dcc0ac840 2024-09-12T02:17:13.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.655+0000 7f0dd37fe640 1 -- 172.21.15.3:0/2812017394 <== osd.9 v2:172.21.15.3:6826/3263587345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0de41021c0 con 0x7f0dcc0ac840 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.655+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] conn(0x7f0dcc0ac840 msgr2=0x7f0dcc0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.655+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] conn(0x7f0dcc0ac840 0x7f0dcc0aece0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0dcc0a2c10 msgr2=0x7f0dcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0dcc0a2c10 0x7f0dcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0dd400abe0 tx=0x7f0dd400a5c0 comp rx=0 tx=0).stop 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 msgr2=0x7f0de41997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 --2- 172.21.15.3:0/2812017394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0de4101420 0x7f0de41997e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0dd802a940 tx=0x7f0dd8003c00 comp rx=0 tx=0).stop 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de2d9d640 1 -- 172.21.15.3:0/2812017394 reap_dead start 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 shutdown_connections 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 >> 172.21.15.3:0/2812017394 conn(0x7f0de40fc7a0 msgr2=0x7f0de40fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 shutdown_connections 2024-09-12T02:17:13.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:12.656+0000 7f0de8c63640 1 -- 172.21.15.3:0/2812017394 wait complete. 2024-09-12T02:17:13.942 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:13 smithi003 ceph-mon[31476]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:17:13.977 INFO:teuthology.orchestra.run.smithi003.stdout:158913790003 2024-09-12T02:17:13.977 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T02:17:14.104 INFO:teuthology.orchestra.run.smithi003.stdout:322122547221 2024-09-12T02:17:14.105 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T02:17:14.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:13 smithi026 ceph-mon[35496]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:17:14.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:13 smithi092 ceph-mon[36089]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:17:14.642 INFO:teuthology.orchestra.run.smithi003.stdout:292057776155 2024-09-12T02:17:14.642 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T02:17:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 -- 172.21.15.3:0/459920249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c103780 msgr2=0x7fe35c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 --2- 172.21.15.3:0/459920249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c103780 0x7fe35c107bc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe350009930 tx=0x7fe35002a420 comp rx=0 tx=0).stop 2024-09-12T02:17:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 -- 172.21.15.3:0/459920249 shutdown_connections 2024-09-12T02:17:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 --2- 172.21.15.3:0/459920249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c108300 0x7fe35c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 --2- 172.21.15.3:0/459920249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c103780 0x7fe35c107bc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 --2- 172.21.15.3:0/459920249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe35c102dd0 0x7fe35c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 -- 172.21.15.3:0/459920249 >> 172.21.15.3:0/459920249 conn(0x7fe35c0fc7d0 msgr2=0x7fe35c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 -- 172.21.15.3:0/459920249 shutdown_connections 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.975+0000 7fe364221640 1 -- 172.21.15.3:0/459920249 wait complete. 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 Processor -- start 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 -- start start 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c102dd0 0x7fe35c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe35c103780 0x7fe35c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 0x7fe35c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1120d0 con 0x7fe35c108300 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c111f50 con 0x7fe35c102dd0 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe364221640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c112250 con 0x7fe35c103780 2024-09-12T02:17:14.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 0x7fe35c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 0x7fe35c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59316/0 (socket says 172.21.15.3:59316) 2024-09-12T02:17:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 -- 172.21.15.3:0/2997957996 learned_addr learned my addr 172.21.15.3:0/2997957996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3617bc640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe35c103780 0x7fe35c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe361fbd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c102dd0 0x7fe35c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:14.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 -- 172.21.15.3:0/2997957996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe35c103780 msgr2=0x7fe35c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe35c103780 0x7fe35c1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 -- 172.21.15.3:0/2997957996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c102dd0 msgr2=0x7fe35c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c102dd0 0x7fe35c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe3627be640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c19af30 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.976+0000 7fe361fbd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35c102dd0 0x7fe35c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe3627be640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 0x7fe35c19a6c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00cad0 tx=0x7fe34c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00ae20 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c005020 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c19b220 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00f6f0 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.977+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c1a73a0 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.978+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe35c062980 con 0x7fe35c108300 2024-09-12T02:17:14.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.979+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe34c0051d0 con 0x7fe35c108300 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe35affd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe3440a2c10 0x7fe3440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe361fbd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe3440a2c10 0x7fe3440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe34c0cf2a0 con 0x7fe35c108300 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe35affd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] conn(0x7fe3440ac840 0x7fe3440aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe34c004810 con 0x7fe3440ac840 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe34c0d6050 con 0x7fe35c108300 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe3617bc640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] conn(0x7fe3440ac840 0x7fe3440aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe361fbd640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe3440a2c10 0x7fe3440a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe354009930 tx=0x7fe354008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.980+0000 7fe3617bc640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] conn(0x7fe3440ac840 0x7fe3440aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.981+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== osd.3 v2:172.21.15.3:6810/278455203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe34c004810 con 0x7fe3440ac840 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 --> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe35c062e30 con 0x7fe3440ac840 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe35affd640 1 -- 172.21.15.3:0/2997957996 <== osd.3 v2:172.21.15.3:6810/278455203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe35c062e30 con 0x7fe3440ac840 2024-09-12T02:17:14.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] conn(0x7fe3440ac840 msgr2=0x7fe3440aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] conn(0x7fe3440ac840 0x7fe3440aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe3440a2c10 msgr2=0x7fe3440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe3440a2c10 0x7fe3440a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe354009930 tx=0x7fe354008040 comp rx=0 tx=0).stop 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 msgr2=0x7fe35c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 --2- 172.21.15.3:0/2997957996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe35c108300 0x7fe35c19a6c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00cad0 tx=0x7fe34c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe3627be640 1 -- 172.21.15.3:0/2997957996 reap_dead start 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 shutdown_connections 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.995+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 >> 172.21.15.3:0/2997957996 conn(0x7fe35c0fc7d0 msgr2=0x7fe35c100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.996+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 shutdown_connections 2024-09-12T02:17:14.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:09.996+0000 7fe364221640 1 -- 172.21.15.3:0/2997957996 wait complete. 2024-09-12T02:17:15.308 INFO:teuthology.orchestra.run.smithi003.stdout:103079215165 2024-09-12T02:17:15.308 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T02:17:16.012 INFO:teuthology.orchestra.run.smithi003.stdout:137438953529 2024-09-12T02:17:16.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T02:17:16.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:15 smithi026 ceph-mon[35496]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:16.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:15 smithi003 ceph-mon[31476]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:16.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:15 smithi092 ceph-mon[36089]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:16.783 INFO:teuthology.orchestra.run.smithi003.stdout:34359738445 2024-09-12T02:17:16.783 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T02:17:16.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- 172.21.15.3:0/4093455873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 msgr2=0x7f4a74101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 --2- 172.21.15.3:0/4093455873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a74101200 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009900 tx=0x7f4a6802a480 comp rx=0 tx=0).stop 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- 172.21.15.3:0/4093455873 shutdown_connections 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 --2- 172.21.15.3:0/4093455873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a74102ef0 0x7f4a7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 --2- 172.21.15.3:0/4093455873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a74102570 0x7f4a741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 --2- 172.21.15.3:0/4093455873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a74101200 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- 172.21.15.3:0/4093455873 >> 172.21.15.3:0/4093455873 conn(0x7f4a740fc7d0 msgr2=0x7f4a740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- 172.21.15.3:0/4093455873 shutdown_connections 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- 172.21.15.3:0/4093455873 wait complete. 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 Processor -- start 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 -- start start 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.951+0000 7f4a7b232640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a7419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:16.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a74102570 0x7f4a7419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a74102ef0 0x7f4a7419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110f80 con 0x7f4a74100e20 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110e00 con 0x7f4a74102570 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74111100 con 0x7f4a74102ef0 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a7419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a7419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59434/0 (socket says 172.21.15.3:59434) 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 -- 172.21.15.3:0/1593544869 learned_addr learned my addr 172.21.15.3:0/1593544869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a73fff640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a74102570 0x7f4a7419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a797cf640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a74102ef0 0x7f4a7419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 -- 172.21.15.3:0/1593544869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a74102ef0 msgr2=0x7f4a7419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a74102ef0 0x7f4a7419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 -- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a74102570 msgr2=0x7f4a7419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a74102570 0x7f4a7419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419f680 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a78fce640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a7419a6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006010 tx=0x7f4a68003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6803f070 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419f910 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a68031e30 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.952+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68035030 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.953+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a7250 con 0x7f4a74100e20 2024-09-12T02:17:16.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.954+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a7406aa60 con 0x7f4a74100e20 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.955+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4a68042070 con 0x7f4a74100e20 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a71ffb640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4a640a2c10 0x7f4a640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a73fff640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4a640a2c10 0x7f4a640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4a680f01a0 con 0x7f4a74100e20 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a71ffb640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] conn(0x7f4a640ac840 0x7f4a640aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a68038e90 con 0x7f4a640ac840 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f4a680f0550 con 0x7f4a74100e20 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a73fff640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4a640a2c10 0x7f4a640a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a740ffe30 tx=0x7f4a6000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.956+0000 7f4a797cf640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] conn(0x7f4a640ac840 0x7f4a640aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.957+0000 7f4a797cf640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] conn(0x7f4a640ac840 0x7f4a640aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:16.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.957+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== osd.1 v2:172.21.15.26:6800/2606408940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a68038e90 con 0x7f4a640ac840 2024-09-12T02:17:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.971+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 --> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a741021b0 con 0x7f4a640ac840 2024-09-12T02:17:16.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a71ffb640 1 -- 172.21.15.3:0/1593544869 <== osd.1 v2:172.21.15.26:6800/2606408940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a741021b0 con 0x7f4a640ac840 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] conn(0x7f4a640ac840 msgr2=0x7f4a640aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] conn(0x7f4a640ac840 0x7f4a640aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4a640a2c10 msgr2=0x7f4a640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4a640a2c10 0x7f4a640a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4a740ffe30 tx=0x7f4a6000a250 comp rx=0 tx=0).stop 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 msgr2=0x7f4a7419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:16.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 --2- 172.21.15.3:0/1593544869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a74100e20 0x7f4a7419a6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a68006010 tx=0x7f4a68003c80 comp rx=0 tx=0).stop 2024-09-12T02:17:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a797cf640 1 -- 172.21.15.3:0/1593544869 reap_dead start 2024-09-12T02:17:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 shutdown_connections 2024-09-12T02:17:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.972+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 >> 172.21.15.3:0/1593544869 conn(0x7f4a740fc7d0 msgr2=0x7f4a7406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.973+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 shutdown_connections 2024-09-12T02:17:16.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:16.973+0000 7f4a7b232640 1 -- 172.21.15.3:0/1593544869 wait complete. 2024-09-12T02:17:17.600 INFO:teuthology.orchestra.run.smithi003.stdout:55834574922 2024-09-12T02:17:17.601 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T02:17:17.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 -- 172.21.15.3:0/364980913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020100e20 msgr2=0x7fc020101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 --2- 172.21.15.3:0/364980913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020100e20 0x7fc020101200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802a480 comp rx=0 tx=0).stop 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 -- 172.21.15.3:0/364980913 shutdown_connections 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 --2- 172.21.15.3:0/364980913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 0x7fc02010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 --2- 172.21.15.3:0/364980913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020102570 0x7fc0201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 --2- 172.21.15.3:0/364980913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020100e20 0x7fc020101200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 -- 172.21.15.3:0/364980913 >> 172.21.15.3:0/364980913 conn(0x7fc0200fc7d0 msgr2=0x7fc0200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 -- 172.21.15.3:0/364980913 shutdown_connections 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.848+0000 7fc026d67640 1 -- 172.21.15.3:0/364980913 wait complete. 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 Processor -- start 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 -- start start 2024-09-12T02:17:17.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020100e20 0x7fc020198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 0x7fc020198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 0x7fc02019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc020110f80 con 0x7fc020102570 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc020110e00 con 0x7fc020100e20 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc026d67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc020111100 con 0x7fc020102ef0 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 0x7fc020198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 0x7fc020198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59464/0 (socket says 172.21.15.3:59464) 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 -- 172.21.15.3:0/3830955954 learned_addr learned my addr 172.21.15.3:0/3830955954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc024b03640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020100e20 0x7fc020198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc025304640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 0x7fc02019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 -- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 msgr2=0x7fc02019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 0x7fc02019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 -- 172.21.15.3:0/3830955954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020100e20 msgr2=0x7fc020198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc020100e20 0x7fc020198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc01ffff640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02019d4b0 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.849+0000 7fc025304640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc020102ef0 0x7fc02019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc01ffff640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 0x7fc020198a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc01000eab0 tx=0x7fc01000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01000ce40 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02019d7a0 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc010011950 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010010ae0 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.850+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a5080 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.851+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc020062980 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.852+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc010011b00 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc01dffb640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc00c0a2c10 0x7fc00c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc0100cf8a0 con 0x7fc020102570 2024-09-12T02:17:17.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc024b03640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc00c0a2c10 0x7fc00c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc01dffb640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] conn(0x7fc00c0ac840 0x7fc00c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc010014ec0 con 0x7fc00c0ac840 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fc0100cfcf0 con 0x7fc020102570 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc024b03640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc00c0a2c10 0x7fc00c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc008009230 tx=0x7fc0080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.853+0000 7fc025304640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] conn(0x7fc00c0ac840 0x7fc00c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.854+0000 7fc025304640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] conn(0x7fc00c0ac840 0x7fc00c0aece0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:17.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.855+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== osd.8 v2:172.21.15.92:6816/1598499134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc010014ec0 con 0x7fc00c0ac840 2024-09-12T02:17:17.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.868+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 --> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc020062e00 con 0x7fc00c0ac840 2024-09-12T02:17:17.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc01dffb640 1 -- 172.21.15.3:0/3830955954 <== osd.8 v2:172.21.15.92:6816/1598499134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc020062e00 con 0x7fc00c0ac840 2024-09-12T02:17:17.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] conn(0x7fc00c0ac840 msgr2=0x7fc00c0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] conn(0x7fc00c0ac840 0x7fc00c0aece0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc00c0a2c10 msgr2=0x7fc00c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc00c0a2c10 0x7fc00c0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc008009230 tx=0x7fc0080058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 msgr2=0x7fc020198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 --2- 172.21.15.3:0/3830955954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc020102570 0x7fc020198a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc01000eab0 tx=0x7fc01000ef70 comp rx=0 tx=0).stop 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc025304640 1 -- 172.21.15.3:0/3830955954 reap_dead start 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.869+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 shutdown_connections 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.870+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 >> 172.21.15.3:0/3830955954 conn(0x7fc0200fc7d0 msgr2=0x7fc02006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.870+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 shutdown_connections 2024-09-12T02:17:17.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:17.870+0000 7fc026d67640 1 -- 172.21.15.3:0/3830955954 wait complete. 2024-09-12T02:17:17.880 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:17 smithi003 ceph-mon[31476]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:18.091 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:17 smithi092 ceph-mon[36089]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:18.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:17 smithi026 ceph-mon[35496]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:17:18.587 INFO:teuthology.orchestra.run.smithi003.stdout:240518168614 2024-09-12T02:17:18.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T02:17:18.971 INFO:teuthology.orchestra.run.smithi003.stdout:270582939680 2024-09-12T02:17:18.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T02:17:19.537 INFO:teuthology.orchestra.run.smithi003.stdout:188978561071 2024-09-12T02:17:19.538 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T02:17:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.593 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.599 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.600 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.605 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.610 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.613 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.616 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.617 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:19.624 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:20.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:19 smithi003 ceph-mon[31476]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:20.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:19 smithi026 ceph-mon[35496]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 -- 172.21.15.3:0/1942279520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8102790 msgr2=0x7fa6a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1942279520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8102790 0x7fa6a81067e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa698009900 tx=0x7fa69802a410 comp rx=0 tx=0).stop 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 -- 172.21.15.3:0/1942279520 shutdown_connections 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1942279520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8106f10 0x7fa6a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1942279520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8102790 0x7fa6a81067e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1942279520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8101de0 0x7fa6a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 -- 172.21.15.3:0/1942279520 >> 172.21.15.3:0/1942279520 conn(0x7fa6a80fd8c0 msgr2=0x7fa6a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 -- 172.21.15.3:0/1942279520 shutdown_connections 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.267+0000 7fa6afe52640 1 -- 172.21.15.3:0/1942279520 wait complete. 2024-09-12T02:17:20.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 Processor -- start 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 -- start start 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 0x7fa6a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8106f10 0x7fa6a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111f70 con 0x7fa6a8101de0 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111df0 con 0x7fa6a8102790 2024-09-12T02:17:20.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6afe52640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81120f0 con 0x7fa6a8106f10 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52908/0 (socket says 172.21.15.3:52908) 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 -- 172.21.15.3:0/1984145996 learned_addr learned my addr 172.21.15.3:0/1984145996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6ae3ef640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8106f10 0x7fa6a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6ad3ed640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 0x7fa6a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 -- 172.21.15.3:0/1984145996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8106f10 msgr2=0x7fa6a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8106f10 0x7fa6a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 -- 172.21.15.3:0/1984145996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 msgr2=0x7fa6a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 0x7fa6a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.268+0000 7fa6adbee640 1 -- 172.21.15.3:0/1984145996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a81a6950 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa6ad3ed640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 0x7fa6a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa6adbee640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c970 tx=0x7fa6a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4007bb0 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4007d50 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a81a6c40 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400f450 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.269+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a7130 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.270+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a81021c0 con 0x7fa6a8101de0 2024-09-12T02:17:20.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.271+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa6a4005020 con 0x7fa6a8101de0 2024-09-12T02:17:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.272+0000 7fa69effd640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 0x7fa6940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.272+0000 7fa6ad3ed640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 0x7fa6940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.272+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa6a40cefd0 con 0x7fa6a8101de0 2024-09-12T02:17:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.272+0000 7fa6ad3ed640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 0x7fa6940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa698000c00 tx=0x7fa698032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:20.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.276+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a4014030 con 0x7fa6a8101de0 2024-09-12T02:17:20.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:19 smithi092 ceph-mon[36089]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:20.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.458+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa6a8109bc0 con 0x7fa6a8101de0 2024-09-12T02:17:20.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.458+0000 7fa69effd640 1 -- 172.21.15.3:0/1984145996 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa6a409bdc0 con 0x7fa6a8101de0 2024-09-12T02:17:20.459 INFO:teuthology.orchestra.run.smithi003.stdout:292057776156 2024-09-12T02:17:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 msgr2=0x7fa6940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 0x7fa6940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa698000c00 tx=0x7fa698032d50 comp rx=0 tx=0).stop 2024-09-12T02:17:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 msgr2=0x7fa6a80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c970 tx=0x7fa6a400ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:20.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 shutdown_connections 2024-09-12T02:17:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa6940a2c10 0x7fa6940a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a8106f10 0x7fa6a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6a8102790 0x7fa6a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 --2- 172.21.15.3:0/1984145996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6a8101de0 0x7fa6a80744f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 >> 172.21.15.3:0/1984145996 conn(0x7fa6a80fd8c0 msgr2=0x7fa6a80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:20.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 shutdown_connections 2024-09-12T02:17:20.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.460+0000 7fa6afe52640 1 -- 172.21.15.3:0/1984145996 wait complete. 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3812703146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca8810e420 msgr2=0x7fca88110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3812703146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca8810e420 0x7fca88110810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fca80009ca0 tx=0x7fca8002a950 comp rx=0 tx=0).stop 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3812703146 shutdown_connections 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3812703146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca8810e420 0x7fca88110810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3812703146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca88074ee0 0x7fca8810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3812703146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca880745c0 0x7fca880749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3812703146 >> 172.21.15.3:0/3812703146 conn(0x7fca880fd8c0 msgr2=0x7fca880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3812703146 shutdown_connections 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.841+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3812703146 wait complete. 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 Processor -- start 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 -- start start 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca880745c0 0x7fca881a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8810e420 0x7fca8819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fca88114230 con 0x7fca88074ee0 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fca881140b0 con 0x7fca880745c0 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8e4d6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca881143b0 con 0x7fca8810e420 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52924/0 (socket says 172.21.15.3:52924) 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca877fe640 1 -- 172.21.15.3:0/3866610560 learned_addr learned my addr 172.21.15.3:0/3866610560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca87fff640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca880745c0 0x7fca881a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.842+0000 7fca8ca73640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8810e420 0x7fca8819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 -- 172.21.15.3:0/3866610560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8810e420 msgr2=0x7fca8819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8810e420 0x7fca8819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 -- 172.21.15.3:0/3866610560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca880745c0 msgr2=0x7fca881a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca880745c0 0x7fca881a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 -- 172.21.15.3:0/3866610560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8819aee0 con 0x7fca88074ee0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca877fe640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00b4a0 tx=0x7fca7c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c015020 con 0x7fca88074ee0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7c011d10 con 0x7fca88074ee0 2024-09-12T02:17:20.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8819b1d0 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00f2e0 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.843+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca881a7520 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.844+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca88075c70 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.845+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fca7c018020 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.846+0000 7fca857fa640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 0x7fca700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.846+0000 7fca87fff640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 0x7fca700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.846+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fca7c0ce8a0 con 0x7fca88074ee0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.847+0000 7fca87fff640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 0x7fca700a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca78009990 tx=0x7fca78009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:20.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:20.851+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca7c01f020 con 0x7fca88074ee0 2024-09-12T02:17:21.023 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:20 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1984145996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:17:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.046+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fca8819bba0 con 0x7fca88074ee0 2024-09-12T02:17:21.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.047+0000 7fca857fa640 1 -- 172.21.15.3:0/3866610560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca7c09b7a0 con 0x7fca88074ee0 2024-09-12T02:17:21.047 INFO:teuthology.orchestra.run.smithi003.stdout:55834574922 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 msgr2=0x7fca700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 0x7fca700a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca78009990 tx=0x7fca78009110 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 msgr2=0x7fca881a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00b4a0 tx=0x7fca7c00b8a0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 shutdown_connections 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fca700a2c90 0x7fca700a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca8810e420 0x7fca8819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca88074ee0 0x7fca881a0ae0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 --2- 172.21.15.3:0/3866610560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca880745c0 0x7fca881a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 >> 172.21.15.3:0/3866610560 conn(0x7fca880fd8c0 msgr2=0x7fca880ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:21.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 shutdown_connections 2024-09-12T02:17:21.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.049+0000 7fca8e4d6640 1 -- 172.21.15.3:0/3866610560 wait complete. 2024-09-12T02:17:21.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:20 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1984145996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:17:21.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:20 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1984145996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:17:21.717 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776155 got 292057776156 for osd.10 2024-09-12T02:17:21.717 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 -- 172.21.15.3:0/772508085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e901036e0 msgr2=0x7f9e90107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 --2- 172.21.15.3:0/772508085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e901036e0 0x7f9e90107b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009930 tx=0x7f9e8402a420 comp rx=0 tx=0).stop 2024-09-12T02:17:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 -- 172.21.15.3:0/772508085 shutdown_connections 2024-09-12T02:17:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 --2- 172.21.15.3:0/772508085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 0x7f9e9010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 --2- 172.21.15.3:0/772508085 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e901036e0 0x7f9e90107b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 --2- 172.21.15.3:0/772508085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e90102d30 0x7f9e90103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 -- 172.21.15.3:0/772508085 >> 172.21.15.3:0/772508085 conn(0x7f9e900fc7d0 msgr2=0x7f9e900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 -- 172.21.15.3:0/772508085 shutdown_connections 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.369+0000 7f9e950d6640 1 -- 172.21.15.3:0/772508085 wait complete. 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 Processor -- start 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 -- start start 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e901036e0 0x7f9e90100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 0x7f9e90101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901120c0 con 0x7f9e90102d30 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90111f40 con 0x7f9e901036e0 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e950d6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90112240 con 0x7f9e901082a0 2024-09-12T02:17:21.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e8ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e8ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52944/0 (socket says 172.21.15.3:52944) 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e8ed9d640 1 -- 172.21.15.3:0/1378306393 learned_addr learned my addr 172.21.15.3:0/1378306393 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e8e59c640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e901036e0 0x7f9e90100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.370+0000 7f9e8f59e640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 0x7f9e90101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 -- 172.21.15.3:0/1378306393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 msgr2=0x7f9e90101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 0x7f9e90101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 -- 172.21.15.3:0/1378306393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e901036e0 msgr2=0x7f9e90100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e901036e0 0x7f9e90100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 -- 172.21.15.3:0/1378306393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e901a6ae0 con 0x7f9e90102d30 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e8ed9d640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000d950 tx=0x7f9e8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80017070 con 0x7f9e90102d30 2024-09-12T02:17:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a6dd0 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e80009c30 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000f6b0 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.371+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a7230 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.372+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90103110 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.373+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9e800109d0 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.374+0000 7f9e77fff640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 0x7f9e780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.374+0000 7f9e8e59c640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 0x7f9e780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.374+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9e800cf200 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.374+0000 7f9e8e59c640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 0x7f9e780a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009900 tx=0x7f9e84032d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.378+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e80009de0 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.557+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f9e9010a2b0 con 0x7f9e90102d30 2024-09-12T02:17:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.558+0000 7f9e77fff640 1 -- 172.21.15.3:0/1378306393 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9e8009bff0 con 0x7f9e90102d30 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stdout:240518168615 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 msgr2=0x7f9e780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 0x7f9e780a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009900 tx=0x7f9e84032d10 comp rx=0 tx=0).stop 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 msgr2=0x7f9e90100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000d950 tx=0x7f9e8000de10 comp rx=0 tx=0).stop 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 shutdown_connections 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9e780a2c10 0x7f9e780a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e901082a0 0x7f9e90101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e901036e0 0x7f9e90100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 --2- 172.21.15.3:0/1378306393 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e90102d30 0x7f9e90100990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 >> 172.21.15.3:0/1378306393 conn(0x7f9e900fc7d0 msgr2=0x7f9e90105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 shutdown_connections 2024-09-12T02:17:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:21.560+0000 7f9e950d6640 1 -- 172.21.15.3:0/1378306393 wait complete. 2024-09-12T02:17:22.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:21 smithi003 ceph-mon[31476]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:22.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:21 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3866610560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:17:22.009 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:21 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1378306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:17:22.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:21 smithi026 ceph-mon[35496]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:22.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:21 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3866610560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:17:22.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:21 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1378306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:17:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:21 smithi092 ceph-mon[36089]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:21 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3866610560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:17:22.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:21 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1378306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:17:22.710 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168614 got 240518168615 for osd.8 2024-09-12T02:17:22.711 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:23.005 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:22 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1246766391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:17:23.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:22 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1246766391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:17:23.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:22 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1246766391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:17:23.329 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574922 for osd.1 2024-09-12T02:17:23.330 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:23.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 -- 172.21.15.3:0/2641687604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624101200 msgr2=0x7fe624101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:23.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 --2- 172.21.15.3:0/2641687604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624101200 0x7fe624101660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe614009900 tx=0x7fe61402a410 comp rx=0 tx=0).stop 2024-09-12T02:17:23.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 -- 172.21.15.3:0/2641687604 shutdown_connections 2024-09-12T02:17:23.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 --2- 172.21.15.3:0/2641687604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 0x7fe62410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 --2- 172.21.15.3:0/2641687604 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624101200 0x7fe624101660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 --2- 172.21.15.3:0/2641687604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624105ad0 0x7fe624105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 -- 172.21.15.3:0/2641687604 >> 172.21.15.3:0/2641687604 conn(0x7fe624077210 msgr2=0x7fe6240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 -- 172.21.15.3:0/2641687604 shutdown_connections 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.009+0000 7fe62be64640 1 -- 172.21.15.3:0/2641687604 wait complete. 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 Processor -- start 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 -- start start 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624101200 0x7fe6241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 0x7fe6241988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe624110ef0 con 0x7fe624105ad0 2024-09-12T02:17:23.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe624110d70 con 0x7fe624101200 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62be64640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe624111070 con 0x7fe624101ba0 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52962/0 (socket says 172.21.15.3:52962) 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 -- 172.21.15.3:0/1246766391 learned_addr learned my addr 172.21.15.3:0/1246766391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe6293ff640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 0x7fe6241988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe629c00640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624101200 0x7fe6241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 -- 172.21.15.3:0/1246766391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 msgr2=0x7fe6241988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 0x7fe6241988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 -- 172.21.15.3:0/1246766391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624101200 msgr2=0x7fe6241983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624101200 0x7fe6241983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 -- 172.21.15.3:0/1246766391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62419d350 con 0x7fe624105ad0 2024-09-12T02:17:23.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.010+0000 7fe62a401640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00cad0 tx=0x7fe61c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.011+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00ae20 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.011+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62419d640 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.011+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe61c004030 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.011+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00f7c0 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.011+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6241a4f20 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.012+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe624101660 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.013+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe61c0041e0 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.014+0000 7fe61affd640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 0x7fe6100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.014+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe61c0cf610 con 0x7fe624105ad0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.014+0000 7fe629c00640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 0x7fe6100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.014+0000 7fe629c00640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 0x7fe6100a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe62000cac0 tx=0x7fe62000c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:23.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.018+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe61c014070 con 0x7fe624105ad0 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stdout:158913790005 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.195+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe6241a52b0 con 0x7fe624105ad0 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.196+0000 7fe61affd640 1 -- 172.21.15.3:0/1246766391 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe61c09c3a0 con 0x7fe624105ad0 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 msgr2=0x7fe6100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 0x7fe6100a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe62000cac0 tx=0x7fe62000c4a0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 msgr2=0x7fe62419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:23.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00cad0 tx=0x7fe61c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 shutdown_connections 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6100a2c10 0x7fe6100a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe624105ad0 0x7fe62419cc70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe624101ba0 0x7fe6241988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 --2- 172.21.15.3:0/1246766391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe624101200 0x7fe6241983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 >> 172.21.15.3:0/1246766391 conn(0x7fe624077210 msgr2=0x7fe6240fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 shutdown_connections 2024-09-12T02:17:23.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:22.198+0000 7fe62be64640 1 -- 172.21.15.3:0/1246766391 wait complete. 2024-09-12T02:17:24.185 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:23 smithi003 ceph-mon[31476]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:17:24.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:23 smithi026 ceph-mon[35496]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:17:24.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:23 smithi092 ceph-mon[36089]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:17:24.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 -- 172.21.15.3:0/1888198483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4106a80 msgr2=0x7fa1c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/1888198483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4106a80 0x7fa1c4106e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009930 tx=0x7fa1c002a420 comp rx=0 tx=0).stop 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 -- 172.21.15.3:0/1888198483 shutdown_connections 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/1888198483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4101420 0x7fa1c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/1888198483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4100a80 0x7fa1c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/1888198483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4106a80 0x7fa1c4106e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 -- 172.21.15.3:0/1888198483 >> 172.21.15.3:0/1888198483 conn(0x7fa1c40fc7a0 msgr2=0x7fa1c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 -- 172.21.15.3:0/1888198483 shutdown_connections 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.505+0000 7fa1cbda4640 1 -- 172.21.15.3:0/1888198483 wait complete. 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 Processor -- start 2024-09-12T02:17:24.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 -- start start 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4101420 0x7fa1c4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 0x7fa1c419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4112030 con 0x7fa1c4100a80 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4111eb0 con 0x7fa1c4106a80 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1cbda4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41121b0 con 0x7fa1c4101420 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.506+0000 7fa1c9b40640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52976/0 (socket says 172.21.15.3:52976) 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 -- 172.21.15.3:0/3396795230 learned_addr learned my addr 172.21.15.3:0/3396795230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c933f640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4101420 0x7fa1c4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1ca341640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 0x7fa1c419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 -- 172.21.15.3:0/3396795230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4101420 msgr2=0x7fa1c4198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4101420 0x7fa1c4198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 -- 172.21.15.3:0/3396795230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 msgr2=0x7fa1c419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 0x7fa1c419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 -- 172.21.15.3:0/3396795230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c419d5e0 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1ca341640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 0x7fa1c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1c9b40640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009a50 tx=0x7fa1c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c003f030 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c419d870 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0003ad0 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0035070 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.507+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a51b0 con 0x7fa1c4100a80 2024-09-12T02:17:24.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.508+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c41021c0 con 0x7fa1c4100a80 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.510+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa1c0042030 con 0x7fa1c4100a80 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.510+0000 7fa1b6ffd640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 0x7fa1b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.510+0000 7fa1c933f640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 0x7fa1b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.510+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa1c00efe40 con 0x7fa1c4100a80 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.511+0000 7fa1c933f640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 0x7fa1b00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40fee70 tx=0x7fa1b8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:24.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.513+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1c00bcc30 con 0x7fa1c4100a80 2024-09-12T02:17:24.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.709+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa1c4108ff0 con 0x7fa1c4100a80 2024-09-12T02:17:24.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.710+0000 7fa1b6ffd640 1 -- 172.21.15.3:0/3396795230 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1c00bc530 con 0x7fa1c4100a80 2024-09-12T02:17:24.710 INFO:teuthology.orchestra.run.smithi003.stdout:73014444102 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 msgr2=0x7fa1b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 0x7fa1b00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40fee70 tx=0x7fa1b8009110 comp rx=0 tx=0).stop 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 msgr2=0x7fa1c4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009a50 tx=0x7fa1c0004270 comp rx=0 tx=0).stop 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 shutdown_connections 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1b00a2c10 0x7fa1b00a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1c4106a80 0x7fa1c419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1c4101420 0x7fa1c4198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 --2- 172.21.15.3:0/3396795230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1c4100a80 0x7fa1c4198630 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 >> 172.21.15.3:0/3396795230 conn(0x7fa1c40fc7a0 msgr2=0x7fa1c40fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 shutdown_connections 2024-09-12T02:17:24.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.712+0000 7fa1cbda4640 1 -- 172.21.15.3:0/3396795230 wait complete. 2024-09-12T02:17:24.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/356007489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4102570 msgr2=0x7fb5c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/356007489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4102570 0x7fb5c41029b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009900 tx=0x7fb5b802a480 comp rx=0 tx=0).stop 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/356007489 shutdown_connections 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/356007489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102ef0 0x7fb5c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/356007489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4102570 0x7fb5c41029b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/356007489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4100e20 0x7fb5c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/356007489 >> 172.21.15.3:0/356007489 conn(0x7fb5c40fc7d0 msgr2=0x7fb5c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/356007489 shutdown_connections 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.752+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/356007489 wait complete. 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 Processor -- start 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 -- start start 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 0x7fb5c41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4102ef0 0x7fb5c419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4110f80 con 0x7fb5c4100e20 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4110e00 con 0x7fb5c4102570 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5cb7f0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4111100 con 0x7fb5c4102ef0 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5c958c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5c958c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53000/0 (socket says 172.21.15.3:53000) 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5c958c640 1 -- 172.21.15.3:0/1827970815 learned_addr learned my addr 172.21.15.3:0/1827970815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5c8d8b640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 0x7fb5c41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.753+0000 7fb5c9d8d640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4102ef0 0x7fb5c419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 -- 172.21.15.3:0/1827970815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4102ef0 msgr2=0x7fb5c419bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4102ef0 0x7fb5c419bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 -- 172.21.15.3:0/1827970815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 msgr2=0x7fb5c41979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 0x7fb5c41979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 -- 172.21.15.3:0/1827970815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c419c420 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c8d8b640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 0x7fb5c41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c958c640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d770 tx=0x7fb5bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc017070 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c419c710 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c4069f70 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5bc005500 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.754+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004e00 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.755+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b0005290 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.757+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb5bc009c30 con 0x7fb5c4100e20 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.758+0000 7fb5c27fc640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 0x7fb5ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:24.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.758+0000 7fb5c8d8b640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 0x7fb5ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.758+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb5bc0ceee0 con 0x7fb5c4100e20 2024-09-12T02:17:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.759+0000 7fb5c8d8b640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 0x7fb5ac0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009230 tx=0x7fb5b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:24.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.761+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5bc09b620 con 0x7fb5c4100e20 2024-09-12T02:17:24.800 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.949+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb5b0005850 con 0x7fb5c4100e20 2024-09-12T02:17:24.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.949+0000 7fb5c27fc640 1 -- 172.21.15.3:0/1827970815 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb5bc0bb070 con 0x7fb5c4100e20 2024-09-12T02:17:24.949 INFO:teuthology.orchestra.run.smithi003.stdout:34359738448 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 msgr2=0x7fb5ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 0x7fb5ac0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009230 tx=0x7fb5b80058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 msgr2=0x7fb5c4197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d770 tx=0x7fb5bc00dc30 comp rx=0 tx=0).stop 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 shutdown_connections 2024-09-12T02:17:24.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb5ac0a2c10 0x7fb5ac0a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5c4102ef0 0x7fb5c419bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5c4102570 0x7fb5c41979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 --2- 172.21.15.3:0/1827970815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb5c4100e20 0x7fb5c4197470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 >> 172.21.15.3:0/1827970815 conn(0x7fb5c40fc7d0 msgr2=0x7fb5c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 shutdown_connections 2024-09-12T02:17:24.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:24.951+0000 7fb5cb7f0640 1 -- 172.21.15.3:0/1827970815 wait complete. 2024-09-12T02:17:25.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:24 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:25.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:24 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3396795230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:17:25.217 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:24 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:25.217 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:24 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3396795230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:17:25.231 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:24 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:17:25.232 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:24 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3396795230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:17:25.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444099 got 73014444102 for osd.2 2024-09-12T02:17:25.440 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:25 smithi003 ceph-mon[31476]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T02:17:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:25 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1827970815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:17:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:25 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3813534041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:17:26.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:25 smithi026 ceph-mon[35496]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T02:17:26.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:25 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1827970815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:17:26.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:25 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3813534041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:17:26.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:25 smithi092 ceph-mon[36089]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-12T02:17:26.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:25 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1827970815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:17:26.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:25 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3813534041' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 -- 172.21.15.3:0/1075208904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 msgr2=0x7fccb410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 --2- 172.21.15.3:0/1075208904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb410e790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fccac009ca0 tx=0x7fccac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 -- 172.21.15.3:0/1075208904 shutdown_connections 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 --2- 172.21.15.3:0/1075208904 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb410e790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 --2- 172.21.15.3:0/1075208904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb41037a0 0x7fccb410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 --2- 172.21.15.3:0/1075208904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb4102df0 0x7fccb41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 -- 172.21.15.3:0/1075208904 >> 172.21.15.3:0/1075208904 conn(0x7fccb40fc7d0 msgr2=0x7fccb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 -- 172.21.15.3:0/1075208904 shutdown_connections 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.666+0000 7fccb8838640 1 -- 172.21.15.3:0/1075208904 wait complete. 2024-09-12T02:17:26.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 Processor -- start 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 -- start start 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb4102df0 0x7fccb4100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 0x7fccb4100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccb41120f0 con 0x7fccb410c2f0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4111f70 con 0x7fccb41037a0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb8838640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fccb4112270 con 0x7fccb4102df0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53066/0 (socket says 172.21.15.3:53066) 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb2d9d640 1 -- 172.21.15.3:0/3798108514 learned_addr learned my addr 172.21.15.3:0/3798108514 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.667+0000 7fccb1d9b640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 0x7fccb4100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb259c640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb4102df0 0x7fccb4100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 -- 172.21.15.3:0/3798108514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb4102df0 msgr2=0x7fccb4100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:26.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb4102df0 0x7fccb4100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 -- 172.21.15.3:0/3798108514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 msgr2=0x7fccb4100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 0x7fccb4100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 -- 172.21.15.3:0/3798108514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb41a6a40 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb1d9b640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 0x7fccb4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb2d9d640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccac031a20 tx=0x7fccac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac003b70 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac003d10 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb41a6cd0 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac02fbd0 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.668+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb41a7260 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.670+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb41031d0 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.671+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fccac02fd70 con 0x7fccb410c2f0 2024-09-12T02:17:26.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.671+0000 7fcca37fe640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 0x7fcc9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.671+0000 7fccb259c640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 0x7fcc9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.671+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fccac0f0180 con 0x7fccb410c2f0 2024-09-12T02:17:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.672+0000 7fccb259c640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 0x7fcc9c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009bd0 tx=0x7fcca4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:26.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.675+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccac0bd230 con 0x7fccb410c2f0 2024-09-12T02:17:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.877+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fccb41072c0 con 0x7fccb410c2f0 2024-09-12T02:17:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.878+0000 7fcca37fe640 1 -- 172.21.15.3:0/3798108514 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fccac0bcb30 con 0x7fccb410c2f0 2024-09-12T02:17:26.878 INFO:teuthology.orchestra.run.smithi003.stdout:270582939682 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 msgr2=0x7fcc9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 0x7fcc9c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009bd0 tx=0x7fcca4005f90 comp rx=0 tx=0).stop 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 msgr2=0x7fccb41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fccac031a20 tx=0x7fccac031a50 comp rx=0 tx=0).stop 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 shutdown_connections 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcc9c0a2c90 0x7fcc9c0a5150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccb410c2f0 0x7fccb41014d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccb41037a0 0x7fccb4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 --2- 172.21.15.3:0/3798108514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fccb4102df0 0x7fccb4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 >> 172.21.15.3:0/3798108514 conn(0x7fccb40fc7d0 msgr2=0x7fccb40fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 shutdown_connections 2024-09-12T02:17:26.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.880+0000 7fccb8838640 1 -- 172.21.15.3:0/3798108514 wait complete. 2024-09-12T02:17:27.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790003 got 158913790005 for osd.5 2024-09-12T02:17:27.192 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:27.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 -- 172.21.15.3:0/1246949029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 msgr2=0x7fbee010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:27.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/1246949029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee010ea90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009ca0 tx=0x7fbed802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 -- 172.21.15.3:0/1246949029 shutdown_connections 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/1246949029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee010ea90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/1246949029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0102790 0x7fbee01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/1246949029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0101de0 0x7fbee01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 -- 172.21.15.3:0/1246949029 >> 172.21.15.3:0/1246949029 conn(0x7fbee00fd8c0 msgr2=0x7fbee00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 -- 172.21.15.3:0/1246949029 shutdown_connections 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.623+0000 7fbee6ce3640 1 -- 172.21.15.3:0/1246949029 wait complete. 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 Processor -- start 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 -- start start 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0101de0 0x7fbee01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0102790 0x7fbee0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:27.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0111f70 con 0x7fbee0106f10 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0111df0 con 0x7fbee0102790 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee6ce3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01120f0 con 0x7fbee0101de0 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee5280640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee5280640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53012/0 (socket says 172.21.15.3:53012) 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.624+0000 7fbee5280640 1 -- 172.21.15.3:0/3813534041 learned_addr learned my addr 172.21.15.3:0/3813534041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbedffff640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0102790 0x7fbee0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee4a7f640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0101de0 0x7fbee01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 -- 172.21.15.3:0/3813534041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0101de0 msgr2=0x7fbee01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0101de0 0x7fbee01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 -- 172.21.15.3:0/3813534041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0102790 msgr2=0x7fbee0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:27.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0102790 0x7fbee0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 -- 172.21.15.3:0/3813534041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee019d480 con 0x7fbee0106f10 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee5280640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbed8031a20 tx=0x7fbed8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003b70 con 0x7fbee0106f10 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee019d710 con 0x7fbee0106f10 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed8003d10 con 0x7fbee0106f10 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed802fc40 con 0x7fbee0106f10 2024-09-12T02:17:27.376 INFO:teuthology.orchestra.run.smithi003.stdout:137438953532 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.625+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a5050 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.627+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee01021c0 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.628+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbed8004050 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.629+0000 7fbeddffb640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 0x7fbed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.629+0000 7fbee4a7f640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 0x7fbed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.629+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbed80f01e0 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.629+0000 7fbee4a7f640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 0x7fbed00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009bd0 tx=0x7fbed4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.633+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbed80bd290 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.816+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbee010a150 con 0x7fbee0106f10 2024-09-12T02:17:27.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.816+0000 7fbeddffb640 1 -- 172.21.15.3:0/3813534041 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbed80bcb90 con 0x7fbee0106f10 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.818+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 msgr2=0x7fbed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 0x7fbed00a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009bd0 tx=0x7fbed4005f90 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 msgr2=0x7fbee019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbed8031a20 tx=0x7fbed8031a50 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 shutdown_connections 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbed00a2c10 0x7fbed00a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee0106f10 0x7fbee019cda0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0102790 0x7fbee0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 --2- 172.21.15.3:0/3813534041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbee0101de0 0x7fbee01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 >> 172.21.15.3:0/3813534041 conn(0x7fbee00fd8c0 msgr2=0x7fbee00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 shutdown_connections 2024-09-12T02:17:27.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:25.819+0000 7fbee6ce3640 1 -- 172.21.15.3:0/3813534041 wait complete. 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2154894822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:17:27.379 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3798108514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2154894822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:17:27.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3798108514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2154894822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:17:27.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3798108514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:17:28.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953529 got 137438953532 for osd.4 2024-09-12T02:17:28.022 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:28.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:28 smithi003 ceph-mon[31476]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 -- 172.21.15.3:0/398975507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8102790 msgr2=0x7f15d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 --2- 172.21.15.3:0/398975507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8102790 0x7f15d81067e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009900 tx=0x7f15c802a410 comp rx=0 tx=0).stop 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 -- 172.21.15.3:0/398975507 shutdown_connections 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 --2- 172.21.15.3:0/398975507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8106f10 0x7f15d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 --2- 172.21.15.3:0/398975507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8102790 0x7f15d81067e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 --2- 172.21.15.3:0/398975507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8101de0 0x7f15d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 -- 172.21.15.3:0/398975507 >> 172.21.15.3:0/398975507 conn(0x7f15d80fd8c0 msgr2=0x7f15d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 -- 172.21.15.3:0/398975507 shutdown_connections 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.554+0000 7f15e0185640 1 -- 172.21.15.3:0/398975507 wait complete. 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 Processor -- start 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 -- start start 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8101de0 0x7f15d81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8102790 0x7f15d81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8111f70 con 0x7f15d8106f10 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f15d8111df0 con 0x7f15d8101de0 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15e0185640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15d81120f0 con 0x7f15d8102790 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15de722640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15de722640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53044/0 (socket says 172.21.15.3:53044) 2024-09-12T02:17:28.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15de722640 1 -- 172.21.15.3:0/2154894822 learned_addr learned my addr 172.21.15.3:0/2154894822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15ddf21640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8101de0 0x7f15d81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.555+0000 7f15dd720640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8102790 0x7f15d81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 -- 172.21.15.3:0/2154894822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8102790 msgr2=0x7f15d81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8102790 0x7f15d81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 -- 172.21.15.3:0/2154894822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8101de0 msgr2=0x7f15d81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8101de0 0x7f15d81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 -- 172.21.15.3:0/2154894822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d819d460 con 0x7f15d8106f10 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15de722640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15d000eaa0 tx=0x7f15d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d000cdb0 con 0x7f15d8106f10 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d0005020 con 0x7f15d8106f10 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d819d750 con 0x7f15d8106f10 2024-09-12T02:17:28.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d00116b0 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.556+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d81a5030 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.557+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d81021c0 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.558+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f15d0005560 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.559+0000 7f15ceffd640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 0x7f15c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.559+0000 7f15ddf21640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 0x7f15c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.559+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15d00cf2b0 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.560+0000 7f15ddf21640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 0x7f15c40a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15d400bbe0 tx=0x7f15d400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.563+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15d0016030 con 0x7f15d8106f10 2024-09-12T02:17:28.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.780+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f15d810a1b0 con 0x7f15d8106f10 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stdout:322122547224 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.780+0000 7f15ceffd640 1 -- 172.21.15.3:0/2154894822 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f15d009ba50 con 0x7f15d8106f10 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 msgr2=0x7f15c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 0x7f15c40a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15d400bbe0 tx=0x7f15d400b390 comp rx=0 tx=0).stop 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 msgr2=0x7f15d819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15d000eaa0 tx=0x7f15d000ef60 comp rx=0 tx=0).stop 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 shutdown_connections 2024-09-12T02:17:28.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15c40a2c90 0x7f15c40a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f15d8106f10 0x7f15d819cd80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d8102790 0x7f15d81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 --2- 172.21.15.3:0/2154894822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15d8101de0 0x7f15d81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 >> 172.21.15.3:0/2154894822 conn(0x7f15d80fd8c0 msgr2=0x7f15d80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 shutdown_connections 2024-09-12T02:17:28.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:26.783+0000 7f15e0185640 1 -- 172.21.15.3:0/2154894822 wait complete. 2024-09-12T02:17:28.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:28 smithi026 ceph-mon[35496]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:17:28.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:28 smithi092 ceph-mon[36089]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:17:29.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547224 for osd.11 2024-09-12T02:17:29.022 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 -- 172.21.15.3:0/4066782396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c102d30 msgr2=0x7fbe0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/4066782396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c102d30 0x7fbe0c103110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009930 tx=0x7fbe0802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 -- 172.21.15.3:0/4066782396 shutdown_connections 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/4066782396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/4066782396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/4066782396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c102d30 0x7fbe0c103110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 -- 172.21.15.3:0/4066782396 >> 172.21.15.3:0/4066782396 conn(0x7fbe0c0fc7d0 msgr2=0x7fbe0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 -- 172.21.15.3:0/4066782396 shutdown_connections 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.161+0000 7fbe13e4d640 1 -- 172.21.15.3:0/4066782396 wait complete. 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 Processor -- start 2024-09-12T02:17:29.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 -- start start 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 0x7fbe0c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c1120a0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c111f20 con 0x7fbe0c1036e0 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe13e4d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0c112220 con 0x7fbe0c102d30 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53798/0 (socket says 172.21.15.3:53798) 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 -- 172.21.15.3:0/3396946830 learned_addr learned my addr 172.21.15.3:0/3396946830 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe113e8640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe11be9640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 0x7fbe0c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 -- 172.21.15.3:0/3396946830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 msgr2=0x7fbe0c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 0x7fbe0c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 -- 172.21.15.3:0/3396946830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 msgr2=0x7fbe0c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 -- 172.21.15.3:0/3396946830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0c19d5d0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe11be9640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 0x7fbe0c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe113e8640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.162+0000 7fbe123ea640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009fd0 tx=0x7fbe0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.163+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04017070 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.163+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe04003e50 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.163+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0c19d8c0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.163+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe040106c0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.163+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe0c1a51a0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.164+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0c06a850 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.165+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbe04003970 con 0x7fbe0c1082a0 2024-09-12T02:17:29.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.166+0000 7fbe02ffd640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 0x7fbdf80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.166+0000 7fbe11be9640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 0x7fbdf80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.166+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbe040cef50 con 0x7fbe0c1082a0 2024-09-12T02:17:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.167+0000 7fbe11be9640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 0x7fbdf80a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe08032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:29.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.170+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe0409bd60 con 0x7fbe0c1082a0 2024-09-12T02:17:29.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.350+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbe0c10a2b0 con 0x7fbe0c1082a0 2024-09-12T02:17:29.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.350+0000 7fbe02ffd640 1 -- 172.21.15.3:0/3396946830 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbe0409b660 con 0x7fbe0c1082a0 2024-09-12T02:17:29.350 INFO:teuthology.orchestra.run.smithi003.stdout:210453397549 2024-09-12T02:17:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 msgr2=0x7fbdf80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 0x7fbdf80a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000c00 tx=0x7fbe08032c70 comp rx=0 tx=0).stop 2024-09-12T02:17:29.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 msgr2=0x7fbe0c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009fd0 tx=0x7fbe0400b260 comp rx=0 tx=0).stop 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 shutdown_connections 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbdf80a2c90 0x7fbdf80a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe0c1082a0 0x7fbe0c19cef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe0c1036e0 0x7fbe0c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 --2- 172.21.15.3:0/3396946830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe0c102d30 0x7fbe0c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.352+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 >> 172.21.15.3:0/3396946830 conn(0x7fbe0c0fc7d0 msgr2=0x7fbe0c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.353+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 shutdown_connections 2024-09-12T02:17:29.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:29.353+0000 7fbe13e4d640 1 -- 172.21.15.3:0/3396946830 wait complete. 2024-09-12T02:17:29.438 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:30.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 -- 172.21.15.3:0/2456506159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4101de0 msgr2=0x7f81e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 --2- 172.21.15.3:0/2456506159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4101de0 0x7f81e41021c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009900 tx=0x7f81d802a410 comp rx=0 tx=0).stop 2024-09-12T02:17:30.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 -- 172.21.15.3:0/2456506159 shutdown_connections 2024-09-12T02:17:30.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 --2- 172.21.15.3:0/2456506159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4106f10 0x7f81e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 --2- 172.21.15.3:0/2456506159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 --2- 172.21.15.3:0/2456506159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4101de0 0x7f81e41021c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 -- 172.21.15.3:0/2456506159 >> 172.21.15.3:0/2456506159 conn(0x7f81e40fd8c0 msgr2=0x7f81e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 -- 172.21.15.3:0/2456506159 shutdown_connections 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.205+0000 7f81eafce640 1 -- 172.21.15.3:0/2456506159 wait complete. 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 Processor -- start 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 -- start start 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 0x7f81e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4111fc0 con 0x7f81e4106f10 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4111e40 con 0x7f81e4101de0 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81eafce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f81e4112140 con 0x7f81e4102790 2024-09-12T02:17:30.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81e956b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81e956b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53818/0 (socket says 172.21.15.3:53818) 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81e956b640 1 -- 172.21.15.3:0/931191818 learned_addr learned my addr 172.21.15.3:0/931191818 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81e8d6a640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 0x7f81e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.206+0000 7f81e3fff640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 -- 172.21.15.3:0/931191818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 msgr2=0x7f81e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 -- 172.21.15.3:0/931191818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 msgr2=0x7f81e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 0x7f81e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 -- 172.21.15.3:0/931191818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e41a6920 con 0x7f81e4106f10 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e8d6a640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 0x7f81e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e3fff640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e956b640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00eb10 tx=0x7f81dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc004b20 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81dc004cc0 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e41a6c10 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc010760 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.207+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e41a7150 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.208+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e41021c0 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.209+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f81dc005020 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.210+0000 7f81e1ffb640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 0x7f81d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.210+0000 7f81e8d6a640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 0x7f81d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.210+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f81dc0cf850 con 0x7f81e4106f10 2024-09-12T02:17:30.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.211+0000 7f81e8d6a640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 0x7f81d40a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005ab0 tx=0x7f81d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:30.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.214+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81dc09c5e0 con 0x7f81e4106f10 2024-09-12T02:17:30.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:30 smithi003 ceph-mon[31476]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:30.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:30 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3396946830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:17:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.394+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f81e410a0f0 con 0x7f81e4106f10 2024-09-12T02:17:30.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.394+0000 7f81e1ffb640 1 -- 172.21.15.3:0/931191818 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f81dc09bee0 con 0x7f81e4106f10 2024-09-12T02:17:30.394 INFO:teuthology.orchestra.run.smithi003.stdout:103079215169 2024-09-12T02:17:30.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 msgr2=0x7f81d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 0x7f81d40a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005ab0 tx=0x7f81d8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 msgr2=0x7f81e4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00eb10 tx=0x7f81dc00c2d0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 shutdown_connections 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f81d40a2c90 0x7f81d40a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81e4106f10 0x7f81e4074f70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4102790 0x7f81e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 --2- 172.21.15.3:0/931191818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81e4101de0 0x7f81e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 >> 172.21.15.3:0/931191818 conn(0x7f81e40fd8c0 msgr2=0x7f81e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.396+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 shutdown_connections 2024-09-12T02:17:30.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:30.397+0000 7f81eafce640 1 -- 172.21.15.3:0/931191818 wait complete. 2024-09-12T02:17:30.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:30 smithi026 ceph-mon[35496]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:30 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3396946830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:17:30.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:30 smithi092 ceph-mon[36089]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:30 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3396946830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:17:30.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738448 for osd.0 2024-09-12T02:17:30.682 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:30.691 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397549 for osd.7 2024-09-12T02:17:30.691 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:31.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:31 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/931191818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:17:31.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:31 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/931191818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:17:31.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:31 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/931191818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:17:31.599 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215165 got 103079215169 for osd.3 2024-09-12T02:17:31.599 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 -- 172.21.15.3:0/633337798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac102d40 msgr2=0x7f78ac103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/633337798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac102d40 0x7f78ac103120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f789c009ca0 tx=0x7f789c02a950 comp rx=0 tx=0).stop 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 -- 172.21.15.3:0/633337798 shutdown_connections 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/633337798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac10c170 0x7f78ac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/633337798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac103660 0x7f78ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/633337798 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac102d40 0x7f78ac103120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 -- 172.21.15.3:0/633337798 >> 172.21.15.3:0/633337798 conn(0x7f78ac0fc7a0 msgr2=0x7f78ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:31.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 -- 172.21.15.3:0/633337798 shutdown_connections 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.788+0000 7f78b0ab5640 1 -- 172.21.15.3:0/633337798 wait complete. 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 Processor -- start 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 -- start start 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac102d40 0x7f78ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac103660 0x7f78ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111fb0 con 0x7f78ac10c170 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111e30 con 0x7f78ac102d40 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.789+0000 7f78b0ab5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac112130 con 0x7f78ac103660 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53826/0 (socket says 172.21.15.3:53826) 2024-09-12T02:17:31.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 -- 172.21.15.3:0/4130968274 learned_addr learned my addr 172.21.15.3:0/4130968274 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78a9d9b640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac103660 0x7f78ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aa59c640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac102d40 0x7f78ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 -- 172.21.15.3:0/4130968274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac103660 msgr2=0x7f78ac1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac103660 0x7f78ac1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 -- 172.21.15.3:0/4130968274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac102d40 msgr2=0x7f78ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac102d40 0x7f78ac198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 -- 172.21.15.3:0/4130968274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac19d430 con 0x7f78ac10c170 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78aad9d640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009fd0 tx=0x7f78a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4011830 con 0x7f78ac10c170 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a4016070 con 0x7f78ac10c170 2024-09-12T02:17:31.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac19d720 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a400f2e0 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.790+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1a5000 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.791+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac103120 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.792+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f78a40119d0 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.793+0000 7f789b7fe640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 0x7f78940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.793+0000 7f78aa59c640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 0x7f78940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.793+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f78a40ce640 con 0x7f78ac10c170 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.794+0000 7f78aa59c640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 0x7f78940a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f789c0093d0 tx=0x7f789c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:31.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.797+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78a409b540 con 0x7f78ac10c170 2024-09-12T02:17:31.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.979+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f78ac106c80 con 0x7f78ac10c170 2024-09-12T02:17:31.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.980+0000 7f789b7fe640 1 -- 172.21.15.3:0/4130968274 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78a409ae40 con 0x7f78ac10c170 2024-09-12T02:17:31.980 INFO:teuthology.orchestra.run.smithi003.stdout:188978561075 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 msgr2=0x7f78940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 0x7f78940a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f789c0093d0 tx=0x7f789c009360 comp rx=0 tx=0).stop 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 msgr2=0x7f78ac19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009fd0 tx=0x7f78a400b8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 shutdown_connections 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f78940a2c10 0x7f78940a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78ac10c170 0x7f78ac19cd50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78ac103660 0x7f78ac1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 --2- 172.21.15.3:0/4130968274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78ac102d40 0x7f78ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:31.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 >> 172.21.15.3:0/4130968274 conn(0x7f78ac0fc7a0 msgr2=0x7f78ac0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:31.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 shutdown_connections 2024-09-12T02:17:31.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:31.982+0000 7f78b0ab5640 1 -- 172.21.15.3:0/4130968274 wait complete. 2024-09-12T02:17:32.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:32 smithi003 ceph-mon[31476]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:32.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:32 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4130968274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:17:32.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939680 got 270582939682 for osd.9 2024-09-12T02:17:32.238 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:32.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:32 smithi026 ceph-mon[35496]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:32 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4130968274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:17:32.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:32 smithi092 ceph-mon[36089]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:32.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:32 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4130968274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:17:32.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561075 for osd.6 2024-09-12T02:17:32.855 DEBUG:teuthology.parallel:result is None 2024-09-12T02:17:32.855 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:17:32.855 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:17:33.683 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 -- 172.21.15.3:0/24045354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102d30 msgr2=0x7f70e4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 --2- 172.21.15.3:0/24045354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102d30 0x7f70e4103110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009cd0 tx=0x7f70d402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 -- 172.21.15.3:0/24045354 shutdown_connections 2024-09-12T02:17:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 --2- 172.21.15.3:0/24045354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e41082a0 0x7f70e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 --2- 172.21.15.3:0/24045354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41036e0 0x7f70e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 --2- 172.21.15.3:0/24045354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e4102d30 0x7f70e4103110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 -- 172.21.15.3:0/24045354 >> 172.21.15.3:0/24045354 conn(0x7f70e40fc7d0 msgr2=0x7f70e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 -- 172.21.15.3:0/24045354 shutdown_connections 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 -- 172.21.15.3:0/24045354 wait complete. 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.211+0000 7f70e9641640 1 Processor -- start 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 -- start start 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102d30 0x7f70e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41082a0 0x7f70e419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4112100 con 0x7f70e41036e0 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4111f80 con 0x7f70e41082a0 2024-09-12T02:17:34.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e9641640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4112280 con 0x7f70e4102d30 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53840/0 (socket says 172.21.15.3:53840) 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 -- 172.21.15.3:0/4129249639 learned_addr learned my addr 172.21.15.3:0/4129249639 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e2ffd640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102d30 0x7f70e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e37fe640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41082a0 0x7f70e419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 -- 172.21.15.3:0/4129249639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102d30 msgr2=0x7f70e4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102d30 0x7f70e4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 -- 172.21.15.3:0/4129249639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41082a0 msgr2=0x7f70e419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41082a0 0x7f70e419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.212+0000 7f70e27fc640 1 -- 172.21.15.3:0/4129249639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e419d5b0 con 0x7f70e41036e0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.213+0000 7f70e27fc640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c970 tx=0x7f70d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.215+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007bb0 con 0x7f70e41036e0 2024-09-12T02:17:34.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.215+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d8007d50 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.215+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d800f450 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.215+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e419d8a0 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.215+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41a5180 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.216+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e406a850 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.217+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f70d8004f60 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.218+0000 7f70cbfff640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 0x7f70cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.218+0000 7f70e2ffd640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 0x7f70cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.218+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f70d80cee60 con 0x7f70e41036e0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.218+0000 7f70e2ffd640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 0x7f70cc0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006010 tx=0x7f70d4009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:34.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.222+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70d8014030 con 0x7f70e41036e0 2024-09-12T02:17:34.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.406+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70e4062630 con 0x7f70cc0a2c90 2024-09-12T02:17:34.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:34 smithi003 ceph-mon[31476]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:34.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.415+0000 7f70cbfff640 1 -- 172.21.15.3:0/4129249639 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258983 (secure 0 0 0) 0x7f70e4062630 con 0x7f70cc0a2c90 2024-09-12T02:17:34.417 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:34.419 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 msgr2=0x7f70cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 0x7f70cc0a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006010 tx=0x7f70d4009900 comp rx=0 tx=0).stop 2024-09-12T02:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 msgr2=0x7f70e4198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:34.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c970 tx=0x7f70d800ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 shutdown_connections 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70cc0a2c90 0x7f70cc0a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e41082a0 0x7f70e419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70e41036e0 0x7f70e4198b40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 --2- 172.21.15.3:0/4129249639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70e4102d30 0x7f70e4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.418+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 >> 172.21.15.3:0/4129249639 conn(0x7f70e40fc7d0 msgr2=0x7f70e4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.419+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 shutdown_connections 2024-09-12T02:17:34.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:34.419+0000 7f70e9641640 1 -- 172.21.15.3:0/4129249639 wait complete. 2024-09-12T02:17:34.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:34 smithi026 ceph-mon[35496]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:34.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:34 smithi092 ceph-mon[36089]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:35.172 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-12T02:17:32.442562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659169,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":665,"num_read_kb":453,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360440,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581384,"statfs":{"total":1151940427776,"available":1151571337216,"internally_reserved":0,"allocated":11452416,"data_stored":7230183,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159326+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:16:38.159326+0000","last_peered":"2024-09-12T02:16:38.159326+0000","last_clean":"2024-09-12T02:16:38.159326+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:16:38.159326+0000","last_undegraded":"2024-09-12T02:16:38.159326+0000","last_fullsized":"2024-09-12T02:16:38.159326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139979+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:16:38.139979+0000","last_peered":"2024-09-12T02:16:38.139979+0000","last_clean":"2024-09-12T02:16:38.139979+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:16:38.139979+0000","last_undegraded":"2024-09-12T02:16:38.139979+0000","last_fullsized":"2024-09-12T02:16:38.139979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140008+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:16:38.140008+0000","last_peered":"2024-09-12T02:16:38.140008+0000","last_clean":"2024-09-12T02:16:38.140008+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:16:38.140008+0000","last_undegraded":"2024-09-12T02:16:38.140008+0000","last_fullsized":"2024-09-12T02:16:38.140008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158570+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:16:38.158570+0000","last_peered":"2024-09-12T02:16:38.158570+0000","last_clean":"2024-09-12T02:16:38.158570+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:16:38.158570+0000","last_undegraded":"2024-09-12T02:16:38.158570+0000","last_fullsized":"2024-09-12T02:16:38.158570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139897+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:16:38.139897+0000","last_peered":"2024-09-12T02:16:38.139897+0000","last_clean":"2024-09-12T02:16:38.139897+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:16:38.139897+0000","last_undegraded":"2024-09-12T02:16:38.139897+0000","last_fullsized":"2024-09-12T02:16:38.139897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156257+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:16:38.156257+0000","last_peered":"2024-09-12T02:16:38.156257+0000","last_clean":"2024-09-12T02:16:38.156257+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:16:38.156257+0000","last_undegraded":"2024-09-12T02:16:38.156257+0000","last_fullsized":"2024-09-12T02:16:38.156257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160088+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:16:38.160088+0000","last_peered":"2024-09-12T02:16:38.160088+0000","last_clean":"2024-09-12T02:16:38.160088+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:16:38.160088+0000","last_undegraded":"2024-09-12T02:16:38.160088+0000","last_fullsized":"2024-09-12T02:16:38.160088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137721+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:16:38.137721+0000","last_peered":"2024-09-12T02:16:38.137721+0000","last_clean":"2024-09-12T02:16:38.137721+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:16:38.137721+0000","last_undegraded":"2024-09-12T02:16:38.137721+0000","last_fullsized":"2024-09-12T02:16:38.137721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136274+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:16:38.136274+0000","last_peered":"2024-09-12T02:16:38.136274+0000","last_clean":"2024-09-12T02:16:38.136274+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:16:38.136274+0000","last_undegraded":"2024-09-12T02:16:38.136274+0000","last_fullsized":"2024-09-12T02:16:38.136274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157082+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:16:38.157082+0000","last_peered":"2024-09-12T02:16:38.157082+0000","last_clean":"2024-09-12T02:16:38.157082+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:16:38.157082+0000","last_undegraded":"2024-09-12T02:16:38.157082+0000","last_fullsized":"2024-09-12T02:16:38.157082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158697+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:16:38.158697+0000","last_peered":"2024-09-12T02:16:38.158697+0000","last_clean":"2024-09-12T02:16:38.158697+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:16:38.158697+0000","last_undegraded":"2024-09-12T02:16:38.158697+0000","last_fullsized":"2024-09-12T02:16:38.158697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157861+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:16:38.157861+0000","last_peered":"2024-09-12T02:16:38.157861+0000","last_clean":"2024-09-12T02:16:38.157861+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:16:38.157861+0000","last_undegraded":"2024-09-12T02:16:38.157861+0000","last_fullsized":"2024-09-12T02:16:38.157861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139935+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:16:38.139935+0000","last_peered":"2024-09-12T02:16:38.139935+0000","last_clean":"2024-09-12T02:16:38.139935+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:16:38.139935+0000","last_undegraded":"2024-09-12T02:16:38.139935+0000","last_fullsized":"2024-09-12T02:16:38.139935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139658+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:16:38.139658+0000","last_peered":"2024-09-12T02:16:38.139658+0000","last_clean":"2024-09-12T02:16:38.139658+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:16:38.139658+0000","last_undegraded":"2024-09-12T02:16:38.139658+0000","last_fullsized":"2024-09-12T02:16:38.139658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135680+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:16:38.135680+0000","last_peered":"2024-09-12T02:16:38.135680+0000","last_clean":"2024-09-12T02:16:38.135680+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:16:38.135680+0000","last_undegraded":"2024-09-12T02:16:38.135680+0000","last_fullsized":"2024-09-12T02:16:38.135680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159427+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:16:38.159427+0000","last_peered":"2024-09-12T02:16:38.159427+0000","last_clean":"2024-09-12T02:16:38.159427+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:16:38.159427+0000","last_undegraded":"2024-09-12T02:16:38.159427+0000","last_fullsized":"2024-09-12T02:16:38.159427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135750+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:16:38.135750+0000","last_peered":"2024-09-12T02:16:38.135750+0000","last_clean":"2024-09-12T02:16:38.135750+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:16:38.135750+0000","last_undegraded":"2024-09-12T02:16:38.135750+0000","last_fullsized":"2024-09-12T02:16:38.135750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142616+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:16:38.142616+0000","last_peered":"2024-09-12T02:16:38.142616+0000","last_clean":"2024-09-12T02:16:38.142616+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:16:38.142616+0000","last_undegraded":"2024-09-12T02:16:38.142616+0000","last_fullsized":"2024-09-12T02:16:38.142616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159081+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:16:38.159081+0000","last_peered":"2024-09-12T02:16:38.159081+0000","last_clean":"2024-09-12T02:16:38.159081+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:16:38.159081+0000","last_undegraded":"2024-09-12T02:16:38.159081+0000","last_fullsized":"2024-09-12T02:16:38.159081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157895+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:16:38.157895+0000","last_peered":"2024-09-12T02:16:38.157895+0000","last_clean":"2024-09-12T02:16:38.157895+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:16:38.157895+0000","last_undegraded":"2024-09-12T02:16:38.157895+0000","last_fullsized":"2024-09-12T02:16:38.157895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135816+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:16:38.135816+0000","last_peered":"2024-09-12T02:16:38.135816+0000","last_clean":"2024-09-12T02:16:38.135816+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:16:38.135816+0000","last_undegraded":"2024-09-12T02:16:38.135816+0000","last_fullsized":"2024-09-12T02:16:38.135816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138128+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:16:38.138128+0000","last_peered":"2024-09-12T02:16:38.138128+0000","last_clean":"2024-09-12T02:16:38.138128+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:16:38.138128+0000","last_undegraded":"2024-09-12T02:16:38.138128+0000","last_fullsized":"2024-09-12T02:16:38.138128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140366+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:16:38.140366+0000","last_peered":"2024-09-12T02:16:38.140366+0000","last_clean":"2024-09-12T02:16:38.140366+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:16:38.140366+0000","last_undegraded":"2024-09-12T02:16:38.140366+0000","last_fullsized":"2024-09-12T02:16:38.140366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138165+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:16:38.138165+0000","last_peered":"2024-09-12T02:16:38.138165+0000","last_clean":"2024-09-12T02:16:38.138165+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:16:38.138165+0000","last_undegraded":"2024-09-12T02:16:38.138165+0000","last_fullsized":"2024-09-12T02:16:38.138165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.143205+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:16:38.143205+0000","last_peered":"2024-09-12T02:16:38.143205+0000","last_clean":"2024-09-12T02:16:38.143205+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:16:38.143205+0000","last_undegraded":"2024-09-12T02:16:38.143205+0000","last_fullsized":"2024-09-12T02:16:38.143205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159095+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:16:38.159095+0000","last_peered":"2024-09-12T02:16:38.159095+0000","last_clean":"2024-09-12T02:16:38.159095+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:16:38.159095+0000","last_undegraded":"2024-09-12T02:16:38.159095+0000","last_fullsized":"2024-09-12T02:16:38.159095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156467+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:16:38.156467+0000","last_peered":"2024-09-12T02:16:38.156467+0000","last_clean":"2024-09-12T02:16:38.156467+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:16:38.156467+0000","last_undegraded":"2024-09-12T02:16:38.156467+0000","last_fullsized":"2024-09-12T02:16:38.156467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159557+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:16:38.159557+0000","last_peered":"2024-09-12T02:16:38.159557+0000","last_clean":"2024-09-12T02:16:38.159557+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:16:38.159557+0000","last_undegraded":"2024-09-12T02:16:38.159557+0000","last_fullsized":"2024-09-12T02:16:38.159557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142741+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:16:38.142741+0000","last_peered":"2024-09-12T02:16:38.142741+0000","last_clean":"2024-09-12T02:16:38.142741+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:16:38.142741+0000","last_undegraded":"2024-09-12T02:16:38.142741+0000","last_fullsized":"2024-09-12T02:16:38.142741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159777+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:16:38.159777+0000","last_peered":"2024-09-12T02:16:38.159777+0000","last_clean":"2024-09-12T02:16:38.159777+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:16:38.159777+0000","last_undegraded":"2024-09-12T02:16:38.159777+0000","last_fullsized":"2024-09-12T02:16:38.159777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158876+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:16:38.158876+0000","last_peered":"2024-09-12T02:16:38.158876+0000","last_clean":"2024-09-12T02:16:38.158876+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:16:38.158876+0000","last_undegraded":"2024-09-12T02:16:38.158876+0000","last_fullsized":"2024-09-12T02:16:38.158876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139696+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:16:38.139696+0000","last_peered":"2024-09-12T02:16:38.139696+0000","last_clean":"2024-09-12T02:16:38.139696+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:16:38.139696+0000","last_undegraded":"2024-09-12T02:16:38.139696+0000","last_fullsized":"2024-09-12T02:16:38.139696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:30.814339+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:17:30.814339+0000","last_peered":"2024-09-12T02:17:30.814339+0000","last_clean":"2024-09-12T02:17:30.814339+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:17:30.814339+0000","last_undegraded":"2024-09-12T02:17:30.814339+0000","last_fullsized":"2024-09-12T02:17:30.814339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140785+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:16:38.140785+0000","last_peered":"2024-09-12T02:16:38.140785+0000","last_clean":"2024-09-12T02:16:38.140785+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:16:38.140785+0000","last_undegraded":"2024-09-12T02:16:38.140785+0000","last_fullsized":"2024-09-12T02:16:38.140785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137776+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:16:38.137776+0000","last_peered":"2024-09-12T02:16:38.137776+0000","last_clean":"2024-09-12T02:16:38.137776+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:16:38.137776+0000","last_undegraded":"2024-09-12T02:16:38.137776+0000","last_fullsized":"2024-09-12T02:16:38.137776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135610+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:16:38.135610+0000","last_peered":"2024-09-12T02:16:38.135610+0000","last_clean":"2024-09-12T02:16:38.135610+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:16:38.135610+0000","last_undegraded":"2024-09-12T02:16:38.135610+0000","last_fullsized":"2024-09-12T02:16:38.135610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.810629+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:17:28.810629+0000","last_peered":"2024-09-12T02:17:28.810629+0000","last_clean":"2024-09-12T02:17:28.810629+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:17:28.810629+0000","last_undegraded":"2024-09-12T02:17:28.810629+0000","last_fullsized":"2024-09-12T02:17:28.810629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136233+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:16:38.136233+0000","last_peered":"2024-09-12T02:16:38.136233+0000","last_clean":"2024-09-12T02:16:38.136233+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:16:38.136233+0000","last_undegraded":"2024-09-12T02:16:38.136233+0000","last_fullsized":"2024-09-12T02:16:38.136233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140363+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:16:38.140363+0000","last_peered":"2024-09-12T02:16:38.140363+0000","last_clean":"2024-09-12T02:16:38.140363+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:16:38.140363+0000","last_undegraded":"2024-09-12T02:16:38.140363+0000","last_fullsized":"2024-09-12T02:16:38.140363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135783+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:16:38.135783+0000","last_peered":"2024-09-12T02:16:38.135783+0000","last_clean":"2024-09-12T02:16:38.135783+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:16:38.135783+0000","last_undegraded":"2024-09-12T02:16:38.135783+0000","last_fullsized":"2024-09-12T02:16:38.135783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'32","reported_seq":162,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.810655+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:17:28.810655+0000","last_peered":"2024-09-12T02:17:28.810655+0000","last_clean":"2024-09-12T02:17:28.810655+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:17:28.810655+0000","last_undegraded":"2024-09-12T02:17:28.810655+0000","last_fullsized":"2024-09-12T02:17:28.810655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":114,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140182+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:16:38.140182+0000","last_peered":"2024-09-12T02:16:38.140182+0000","last_clean":"2024-09-12T02:16:38.140182+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:16:38.140182+0000","last_undegraded":"2024-09-12T02:16:38.140182+0000","last_fullsized":"2024-09-12T02:16:38.140182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157126+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:16:38.157126+0000","last_peered":"2024-09-12T02:16:38.157126+0000","last_clean":"2024-09-12T02:16:38.157126+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:16:38.157126+0000","last_undegraded":"2024-09-12T02:16:38.157126+0000","last_fullsized":"2024-09-12T02:16:38.157126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160799+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:16:38.160799+0000","last_peered":"2024-09-12T02:16:38.160799+0000","last_clean":"2024-09-12T02:16:38.160799+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:16:38.160799+0000","last_undegraded":"2024-09-12T02:16:38.160799+0000","last_fullsized":"2024-09-12T02:16:38.160799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:25.806377+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:17:25.806377+0000","last_peered":"2024-09-12T02:17:25.806377+0000","last_clean":"2024-09-12T02:17:25.806377+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:17:25.806377+0000","last_undegraded":"2024-09-12T02:17:25.806377+0000","last_fullsized":"2024-09-12T02:17:25.806377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137650+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:16:38.137650+0000","last_peered":"2024-09-12T02:16:38.137650+0000","last_clean":"2024-09-12T02:16:38.137650+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:16:38.137650+0000","last_undegraded":"2024-09-12T02:16:38.137650+0000","last_fullsized":"2024-09-12T02:16:38.137650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142089+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:16:38.142089+0000","last_peered":"2024-09-12T02:16:38.142089+0000","last_clean":"2024-09-12T02:16:38.142089+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:16:38.142089+0000","last_undegraded":"2024-09-12T02:16:38.142089+0000","last_fullsized":"2024-09-12T02:16:38.142089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739199+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:16:39.739199+0000","last_peered":"2024-09-12T02:16:39.739199+0000","last_clean":"2024-09-12T02:16:39.739199+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:16:39.739199+0000","last_undegraded":"2024-09-12T02:16:39.739199+0000","last_fullsized":"2024-09-12T02:16:39.739199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159706+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:16:38.159706+0000","last_peered":"2024-09-12T02:16:38.159706+0000","last_clean":"2024-09-12T02:16:38.159706+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:16:38.159706+0000","last_undegraded":"2024-09-12T02:16:38.159706+0000","last_fullsized":"2024-09-12T02:16:38.159706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138164+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:16:38.138164+0000","last_peered":"2024-09-12T02:16:38.138164+0000","last_clean":"2024-09-12T02:16:38.138164+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:16:38.138164+0000","last_undegraded":"2024-09-12T02:16:38.138164+0000","last_fullsized":"2024-09-12T02:16:38.138164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:29.812695+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:17:29.812695+0000","last_peered":"2024-09-12T02:17:29.812695+0000","last_clean":"2024-09-12T02:17:29.812695+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:17:29.812695+0000","last_undegraded":"2024-09-12T02:17:29.812695+0000","last_fullsized":"2024-09-12T02:17:29.812695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138219+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:16:38.138219+0000","last_peered":"2024-09-12T02:16:38.138219+0000","last_clean":"2024-09-12T02:16:38.138219+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:16:38.138219+0000","last_undegraded":"2024-09-12T02:16:38.138219+0000","last_fullsized":"2024-09-12T02:16:38.138219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137830+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137830+0000","last_peered":"2024-09-12T02:16:38.137830+0000","last_clean":"2024-09-12T02:16:38.137830+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137830+0000","last_undegraded":"2024-09-12T02:16:38.137830+0000","last_fullsized":"2024-09-12T02:16:38.137830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138042+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:16:38.138042+0000","last_peered":"2024-09-12T02:16:38.138042+0000","last_clean":"2024-09-12T02:16:38.138042+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:16:38.138042+0000","last_undegraded":"2024-09-12T02:16:38.138042+0000","last_fullsized":"2024-09-12T02:16:38.138042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157616+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:16:38.157616+0000","last_peered":"2024-09-12T02:16:38.157616+0000","last_clean":"2024-09-12T02:16:38.157616+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:16:38.157616+0000","last_undegraded":"2024-09-12T02:16:38.157616+0000","last_fullsized":"2024-09-12T02:16:38.157616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:26.806979+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:17:26.806979+0000","last_peered":"2024-09-12T02:17:26.806979+0000","last_clean":"2024-09-12T02:17:26.806979+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:17:26.806979+0000","last_undegraded":"2024-09-12T02:17:26.806979+0000","last_fullsized":"2024-09-12T02:17:26.806979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140764+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:16:38.140764+0000","last_peered":"2024-09-12T02:16:38.140764+0000","last_clean":"2024-09-12T02:16:38.140764+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:16:38.140764+0000","last_undegraded":"2024-09-12T02:16:38.140764+0000","last_fullsized":"2024-09-12T02:16:38.140764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:26.807394+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:17:26.807394+0000","last_peered":"2024-09-12T02:17:26.807394+0000","last_clean":"2024-09-12T02:17:26.807394+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:17:26.807394+0000","last_undegraded":"2024-09-12T02:17:26.807394+0000","last_fullsized":"2024-09-12T02:17:26.807394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156353+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:16:38.156353+0000","last_peered":"2024-09-12T02:16:38.156353+0000","last_clean":"2024-09-12T02:16:38.156353+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:16:38.156353+0000","last_undegraded":"2024-09-12T02:16:38.156353+0000","last_fullsized":"2024-09-12T02:16:38.156353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156722+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:16:38.156722+0000","last_peered":"2024-09-12T02:16:38.156722+0000","last_clean":"2024-09-12T02:16:38.156722+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:16:38.156722+0000","last_undegraded":"2024-09-12T02:16:38.156722+0000","last_fullsized":"2024-09-12T02:16:38.156722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156574+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:16:38.156574+0000","last_peered":"2024-09-12T02:16:38.156574+0000","last_clean":"2024-09-12T02:16:38.156574+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:16:38.156574+0000","last_undegraded":"2024-09-12T02:16:38.156574+0000","last_fullsized":"2024-09-12T02:16:38.156574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:55.125142+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:55.125142+0000","last_peered":"2024-09-12T02:16:55.125142+0000","last_clean":"2024-09-12T02:16:55.125142+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:55.125142+0000","last_undegraded":"2024-09-12T02:16:55.125142+0000","last_fullsized":"2024-09-12T02:16:55.125142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138258+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:16:38.138258+0000","last_peered":"2024-09-12T02:16:38.138258+0000","last_clean":"2024-09-12T02:16:38.138258+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:16:38.138258+0000","last_undegraded":"2024-09-12T02:16:38.138258+0000","last_fullsized":"2024-09-12T02:16:38.138258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.161245+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:16:38.161245+0000","last_peered":"2024-09-12T02:16:38.161245+0000","last_clean":"2024-09-12T02:16:38.161245+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:16:38.161245+0000","last_undegraded":"2024-09-12T02:16:38.161245+0000","last_fullsized":"2024-09-12T02:16:38.161245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139845+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:16:38.139845+0000","last_peered":"2024-09-12T02:16:38.139845+0000","last_clean":"2024-09-12T02:16:38.139845+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:16:38.139845+0000","last_undegraded":"2024-09-12T02:16:38.139845+0000","last_fullsized":"2024-09-12T02:16:38.139845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159016+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:16:38.159016+0000","last_peered":"2024-09-12T02:16:38.159016+0000","last_clean":"2024-09-12T02:16:38.159016+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:16:38.159016+0000","last_undegraded":"2024-09-12T02:16:38.159016+0000","last_fullsized":"2024-09-12T02:16:38.159016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158385+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:16:38.158385+0000","last_peered":"2024-09-12T02:16:38.158385+0000","last_clean":"2024-09-12T02:16:38.158385+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:16:38.158385+0000","last_undegraded":"2024-09-12T02:16:38.158385+0000","last_fullsized":"2024-09-12T02:16:38.158385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137813+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:16:38.137813+0000","last_peered":"2024-09-12T02:16:38.137813+0000","last_clean":"2024-09-12T02:16:38.137813+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:16:38.137813+0000","last_undegraded":"2024-09-12T02:16:38.137813+0000","last_fullsized":"2024-09-12T02:16:38.137813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140393+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:16:38.140393+0000","last_peered":"2024-09-12T02:16:38.140393+0000","last_clean":"2024-09-12T02:16:38.140393+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:16:38.140393+0000","last_undegraded":"2024-09-12T02:16:38.140393+0000","last_fullsized":"2024-09-12T02:16:38.140393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135916+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:16:38.135916+0000","last_peered":"2024-09-12T02:16:38.135916+0000","last_clean":"2024-09-12T02:16:38.135916+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:16:38.135916+0000","last_undegraded":"2024-09-12T02:16:38.135916+0000","last_fullsized":"2024-09-12T02:16:38.135916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140265+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:16:38.140265+0000","last_peered":"2024-09-12T02:16:38.140265+0000","last_clean":"2024-09-12T02:16:38.140265+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:16:38.140265+0000","last_undegraded":"2024-09-12T02:16:38.140265+0000","last_fullsized":"2024-09-12T02:16:38.140265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138254+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:16:38.138254+0000","last_peered":"2024-09-12T02:16:38.138254+0000","last_clean":"2024-09-12T02:16:38.138254+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:16:38.138254+0000","last_undegraded":"2024-09-12T02:16:38.138254+0000","last_fullsized":"2024-09-12T02:16:38.138254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.809894+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:17:28.809894+0000","last_peered":"2024-09-12T02:17:28.809894+0000","last_clean":"2024-09-12T02:17:28.809894+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:17:28.809894+0000","last_undegraded":"2024-09-12T02:17:28.809894+0000","last_fullsized":"2024-09-12T02:17:28.809894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68567,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":539,"num_read_kb":344,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1091696,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1031743,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438955,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438423,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":501447,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":501620,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:17:35.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:17:35.439 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:35.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:35 smithi026 ceph-mon[35496]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:35.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:35 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:35 smithi003 ceph-mon[31476]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:35.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:35 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:35 smithi092 ceph-mon[36089]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:35 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:35.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 -- 172.21.15.3:0/120858374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c101420 msgr2=0x7f472c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 --2- 172.21.15.3:0/120858374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c101420 0x7f472c10d950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4724009cd0 tx=0x7f472402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 -- 172.21.15.3:0/120858374 shutdown_connections 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 --2- 172.21.15.3:0/120858374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c101420 0x7f472c10d950 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 --2- 172.21.15.3:0/120858374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c100a80 0x7f472c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 --2- 172.21.15.3:0/120858374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c106a80 0x7f472c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 -- 172.21.15.3:0/120858374 >> 172.21.15.3:0/120858374 conn(0x7f472c0fc7a0 msgr2=0x7f472c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 -- 172.21.15.3:0/120858374 shutdown_connections 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.920+0000 7f47320dd640 1 -- 172.21.15.3:0/120858374 wait complete. 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 Processor -- start 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 -- start start 2024-09-12T02:17:35.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c100a80 0x7f472c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c101420 0x7f472c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f472c111fa0 con 0x7f472c106a80 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f472c111e20 con 0x7f472c100a80 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.921+0000 7f47320dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f472c112120 con 0x7f472c101420 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53844/0 (socket says 172.21.15.3:53844) 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 -- 172.21.15.3:0/102570954 learned_addr learned my addr 172.21.15.3:0/102570954 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:35.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472b7fe640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c100a80 0x7f472c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472affd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c101420 0x7f472c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 -- 172.21.15.3:0/102570954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c101420 msgr2=0x7f472c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c101420 0x7f472c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 -- 172.21.15.3:0/102570954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c100a80 msgr2=0x7f472c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c100a80 0x7f472c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 -- 172.21.15.3:0/102570954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19d3b0 con 0x7f472c106a80 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f472bfff640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4724006010 tx=0x7f4724003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004450 con 0x7f472c106a80 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4724003e10 con 0x7f472c106a80 2024-09-12T02:17:35.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19d640 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724046480 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.922+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a4f80 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.923+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c100ee0 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.924+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f47240045f0 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.925+0000 7f4728ff9640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 0x7f47140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.925+0000 7f472b7fe640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 0x7f47140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.925+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f472402e030 con 0x7f472c106a80 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.926+0000 7f472b7fe640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 0x7f47140a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f471c00abe0 tx=0x7f471c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:35.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:35.929+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f472404b060 con 0x7f472c106a80 2024-09-12T02:17:36.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.103+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f472c0fd4d0 con 0x7f47140a2c90 2024-09-12T02:17:36.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.109+0000 7f4728ff9640 1 -- 172.21.15.3:0/102570954 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258983 (secure 0 0 0) 0x7f472c0fd4d0 con 0x7f47140a2c90 2024-09-12T02:17:36.111 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:36.112 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:17:36.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 msgr2=0x7f47140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 0x7f47140a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f471c00abe0 tx=0x7f471c00a2d0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 msgr2=0x7f472c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4724006010 tx=0x7f4724003c20 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 shutdown_connections 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47140a2c90 0x7f47140a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f472c106a80 0x7f472c19ccd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c101420 0x7f472c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 --2- 172.21.15.3:0/102570954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f472c100a80 0x7f472c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 >> 172.21.15.3:0/102570954 conn(0x7f472c0fc7a0 msgr2=0x7f472c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.113+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 shutdown_connections 2024-09-12T02:17:36.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:36.114+0000 7f47320dd640 1 -- 172.21.15.3:0/102570954 wait complete. 2024-09-12T02:17:36.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:36 smithi003 ceph-mon[31476]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:36.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:36.408 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:36.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:36 smithi026 ceph-mon[35496]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:36.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:36 smithi092 ceph-mon[36089]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:17:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:17:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:17:36.748 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-12T02:17:34.442971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659169,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":680,"num_read_kb":458,"num_write":426,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360440,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581384,"statfs":{"total":1151940427776,"available":1151571337216,"internally_reserved":0,"allocated":11452416,"data_stored":7230183,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002805"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159326+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:16:38.159326+0000","last_peered":"2024-09-12T02:16:38.159326+0000","last_clean":"2024-09-12T02:16:38.159326+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:16:38.159326+0000","last_undegraded":"2024-09-12T02:16:38.159326+0000","last_fullsized":"2024-09-12T02:16:38.159326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139979+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:16:38.139979+0000","last_peered":"2024-09-12T02:16:38.139979+0000","last_clean":"2024-09-12T02:16:38.139979+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:16:38.139979+0000","last_undegraded":"2024-09-12T02:16:38.139979+0000","last_fullsized":"2024-09-12T02:16:38.139979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140008+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:16:38.140008+0000","last_peered":"2024-09-12T02:16:38.140008+0000","last_clean":"2024-09-12T02:16:38.140008+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:16:38.140008+0000","last_undegraded":"2024-09-12T02:16:38.140008+0000","last_fullsized":"2024-09-12T02:16:38.140008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158570+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:16:38.158570+0000","last_peered":"2024-09-12T02:16:38.158570+0000","last_clean":"2024-09-12T02:16:38.158570+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:16:38.158570+0000","last_undegraded":"2024-09-12T02:16:38.158570+0000","last_fullsized":"2024-09-12T02:16:38.158570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139897+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:16:38.139897+0000","last_peered":"2024-09-12T02:16:38.139897+0000","last_clean":"2024-09-12T02:16:38.139897+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:16:38.139897+0000","last_undegraded":"2024-09-12T02:16:38.139897+0000","last_fullsized":"2024-09-12T02:16:38.139897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156257+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:16:38.156257+0000","last_peered":"2024-09-12T02:16:38.156257+0000","last_clean":"2024-09-12T02:16:38.156257+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:16:38.156257+0000","last_undegraded":"2024-09-12T02:16:38.156257+0000","last_fullsized":"2024-09-12T02:16:38.156257+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160088+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:16:38.160088+0000","last_peered":"2024-09-12T02:16:38.160088+0000","last_clean":"2024-09-12T02:16:38.160088+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:16:38.160088+0000","last_undegraded":"2024-09-12T02:16:38.160088+0000","last_fullsized":"2024-09-12T02:16:38.160088+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137721+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:16:38.137721+0000","last_peered":"2024-09-12T02:16:38.137721+0000","last_clean":"2024-09-12T02:16:38.137721+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:16:38.137721+0000","last_undegraded":"2024-09-12T02:16:38.137721+0000","last_fullsized":"2024-09-12T02:16:38.137721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136274+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:16:38.136274+0000","last_peered":"2024-09-12T02:16:38.136274+0000","last_clean":"2024-09-12T02:16:38.136274+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:16:38.136274+0000","last_undegraded":"2024-09-12T02:16:38.136274+0000","last_fullsized":"2024-09-12T02:16:38.136274+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157082+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:16:38.157082+0000","last_peered":"2024-09-12T02:16:38.157082+0000","last_clean":"2024-09-12T02:16:38.157082+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:16:38.157082+0000","last_undegraded":"2024-09-12T02:16:38.157082+0000","last_fullsized":"2024-09-12T02:16:38.157082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158697+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:16:38.158697+0000","last_peered":"2024-09-12T02:16:38.158697+0000","last_clean":"2024-09-12T02:16:38.158697+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:16:38.158697+0000","last_undegraded":"2024-09-12T02:16:38.158697+0000","last_fullsized":"2024-09-12T02:16:38.158697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157861+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:16:38.157861+0000","last_peered":"2024-09-12T02:16:38.157861+0000","last_clean":"2024-09-12T02:16:38.157861+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:16:38.157861+0000","last_undegraded":"2024-09-12T02:16:38.157861+0000","last_fullsized":"2024-09-12T02:16:38.157861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139935+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:16:38.139935+0000","last_peered":"2024-09-12T02:16:38.139935+0000","last_clean":"2024-09-12T02:16:38.139935+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:16:38.139935+0000","last_undegraded":"2024-09-12T02:16:38.139935+0000","last_fullsized":"2024-09-12T02:16:38.139935+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139658+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:16:38.139658+0000","last_peered":"2024-09-12T02:16:38.139658+0000","last_clean":"2024-09-12T02:16:38.139658+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:16:38.139658+0000","last_undegraded":"2024-09-12T02:16:38.139658+0000","last_fullsized":"2024-09-12T02:16:38.139658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135680+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:16:38.135680+0000","last_peered":"2024-09-12T02:16:38.135680+0000","last_clean":"2024-09-12T02:16:38.135680+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:16:38.135680+0000","last_undegraded":"2024-09-12T02:16:38.135680+0000","last_fullsized":"2024-09-12T02:16:38.135680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159427+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:16:38.159427+0000","last_peered":"2024-09-12T02:16:38.159427+0000","last_clean":"2024-09-12T02:16:38.159427+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:16:38.159427+0000","last_undegraded":"2024-09-12T02:16:38.159427+0000","last_fullsized":"2024-09-12T02:16:38.159427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135750+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:16:38.135750+0000","last_peered":"2024-09-12T02:16:38.135750+0000","last_clean":"2024-09-12T02:16:38.135750+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:16:38.135750+0000","last_undegraded":"2024-09-12T02:16:38.135750+0000","last_fullsized":"2024-09-12T02:16:38.135750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142616+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:16:38.142616+0000","last_peered":"2024-09-12T02:16:38.142616+0000","last_clean":"2024-09-12T02:16:38.142616+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:16:38.142616+0000","last_undegraded":"2024-09-12T02:16:38.142616+0000","last_fullsized":"2024-09-12T02:16:38.142616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159081+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:16:38.159081+0000","last_peered":"2024-09-12T02:16:38.159081+0000","last_clean":"2024-09-12T02:16:38.159081+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:16:38.159081+0000","last_undegraded":"2024-09-12T02:16:38.159081+0000","last_fullsized":"2024-09-12T02:16:38.159081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157895+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:16:38.157895+0000","last_peered":"2024-09-12T02:16:38.157895+0000","last_clean":"2024-09-12T02:16:38.157895+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:16:38.157895+0000","last_undegraded":"2024-09-12T02:16:38.157895+0000","last_fullsized":"2024-09-12T02:16:38.157895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135816+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:16:38.135816+0000","last_peered":"2024-09-12T02:16:38.135816+0000","last_clean":"2024-09-12T02:16:38.135816+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:16:38.135816+0000","last_undegraded":"2024-09-12T02:16:38.135816+0000","last_fullsized":"2024-09-12T02:16:38.135816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138128+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:16:38.138128+0000","last_peered":"2024-09-12T02:16:38.138128+0000","last_clean":"2024-09-12T02:16:38.138128+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:16:38.138128+0000","last_undegraded":"2024-09-12T02:16:38.138128+0000","last_fullsized":"2024-09-12T02:16:38.138128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140366+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:16:38.140366+0000","last_peered":"2024-09-12T02:16:38.140366+0000","last_clean":"2024-09-12T02:16:38.140366+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:16:38.140366+0000","last_undegraded":"2024-09-12T02:16:38.140366+0000","last_fullsized":"2024-09-12T02:16:38.140366+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138165+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:16:38.138165+0000","last_peered":"2024-09-12T02:16:38.138165+0000","last_clean":"2024-09-12T02:16:38.138165+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:16:38.138165+0000","last_undegraded":"2024-09-12T02:16:38.138165+0000","last_fullsized":"2024-09-12T02:16:38.138165+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.143205+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:16:38.143205+0000","last_peered":"2024-09-12T02:16:38.143205+0000","last_clean":"2024-09-12T02:16:38.143205+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:16:38.143205+0000","last_undegraded":"2024-09-12T02:16:38.143205+0000","last_fullsized":"2024-09-12T02:16:38.143205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159095+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:16:38.159095+0000","last_peered":"2024-09-12T02:16:38.159095+0000","last_clean":"2024-09-12T02:16:38.159095+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:16:38.159095+0000","last_undegraded":"2024-09-12T02:16:38.159095+0000","last_fullsized":"2024-09-12T02:16:38.159095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156467+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:16:38.156467+0000","last_peered":"2024-09-12T02:16:38.156467+0000","last_clean":"2024-09-12T02:16:38.156467+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:16:38.156467+0000","last_undegraded":"2024-09-12T02:16:38.156467+0000","last_fullsized":"2024-09-12T02:16:38.156467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159557+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:16:38.159557+0000","last_peered":"2024-09-12T02:16:38.159557+0000","last_clean":"2024-09-12T02:16:38.159557+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:16:38.159557+0000","last_undegraded":"2024-09-12T02:16:38.159557+0000","last_fullsized":"2024-09-12T02:16:38.159557+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142741+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:16:38.142741+0000","last_peered":"2024-09-12T02:16:38.142741+0000","last_clean":"2024-09-12T02:16:38.142741+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:16:38.142741+0000","last_undegraded":"2024-09-12T02:16:38.142741+0000","last_fullsized":"2024-09-12T02:16:38.142741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159777+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:16:38.159777+0000","last_peered":"2024-09-12T02:16:38.159777+0000","last_clean":"2024-09-12T02:16:38.159777+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:16:38.159777+0000","last_undegraded":"2024-09-12T02:16:38.159777+0000","last_fullsized":"2024-09-12T02:16:38.159777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158876+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:16:38.158876+0000","last_peered":"2024-09-12T02:16:38.158876+0000","last_clean":"2024-09-12T02:16:38.158876+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:16:38.158876+0000","last_undegraded":"2024-09-12T02:16:38.158876+0000","last_fullsized":"2024-09-12T02:16:38.158876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139696+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:16:38.139696+0000","last_peered":"2024-09-12T02:16:38.139696+0000","last_clean":"2024-09-12T02:16:38.139696+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:16:38.139696+0000","last_undegraded":"2024-09-12T02:16:38.139696+0000","last_fullsized":"2024-09-12T02:16:38.139696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:30.814339+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:17:30.814339+0000","last_peered":"2024-09-12T02:17:30.814339+0000","last_clean":"2024-09-12T02:17:30.814339+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:17:30.814339+0000","last_undegraded":"2024-09-12T02:17:30.814339+0000","last_fullsized":"2024-09-12T02:17:30.814339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140785+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:16:38.140785+0000","last_peered":"2024-09-12T02:16:38.140785+0000","last_clean":"2024-09-12T02:16:38.140785+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:16:38.140785+0000","last_undegraded":"2024-09-12T02:16:38.140785+0000","last_fullsized":"2024-09-12T02:16:38.140785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137776+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:16:38.137776+0000","last_peered":"2024-09-12T02:16:38.137776+0000","last_clean":"2024-09-12T02:16:38.137776+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:16:38.137776+0000","last_undegraded":"2024-09-12T02:16:38.137776+0000","last_fullsized":"2024-09-12T02:16:38.137776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135610+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:16:38.135610+0000","last_peered":"2024-09-12T02:16:38.135610+0000","last_clean":"2024-09-12T02:16:38.135610+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:16:38.135610+0000","last_undegraded":"2024-09-12T02:16:38.135610+0000","last_fullsized":"2024-09-12T02:16:38.135610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.810629+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:17:28.810629+0000","last_peered":"2024-09-12T02:17:28.810629+0000","last_clean":"2024-09-12T02:17:28.810629+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:17:28.810629+0000","last_undegraded":"2024-09-12T02:17:28.810629+0000","last_fullsized":"2024-09-12T02:17:28.810629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.136233+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:16:38.136233+0000","last_peered":"2024-09-12T02:16:38.136233+0000","last_clean":"2024-09-12T02:16:38.136233+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:16:38.136233+0000","last_undegraded":"2024-09-12T02:16:38.136233+0000","last_fullsized":"2024-09-12T02:16:38.136233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140363+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:16:38.140363+0000","last_peered":"2024-09-12T02:16:38.140363+0000","last_clean":"2024-09-12T02:16:38.140363+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:16:38.140363+0000","last_undegraded":"2024-09-12T02:16:38.140363+0000","last_fullsized":"2024-09-12T02:16:38.140363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135783+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:16:38.135783+0000","last_peered":"2024-09-12T02:16:38.135783+0000","last_clean":"2024-09-12T02:16:38.135783+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:16:38.135783+0000","last_undegraded":"2024-09-12T02:16:38.135783+0000","last_fullsized":"2024-09-12T02:16:38.135783+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'32","reported_seq":162,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.810655+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:17:28.810655+0000","last_peered":"2024-09-12T02:17:28.810655+0000","last_clean":"2024-09-12T02:17:28.810655+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:17:28.810655+0000","last_undegraded":"2024-09-12T02:17:28.810655+0000","last_fullsized":"2024-09-12T02:17:28.810655+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61752,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":114,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140182+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:16:38.140182+0000","last_peered":"2024-09-12T02:16:38.140182+0000","last_clean":"2024-09-12T02:16:38.140182+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:16:38.140182+0000","last_undegraded":"2024-09-12T02:16:38.140182+0000","last_fullsized":"2024-09-12T02:16:38.140182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157126+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:16:38.157126+0000","last_peered":"2024-09-12T02:16:38.157126+0000","last_clean":"2024-09-12T02:16:38.157126+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:16:38.157126+0000","last_undegraded":"2024-09-12T02:16:38.157126+0000","last_fullsized":"2024-09-12T02:16:38.157126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.160799+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:16:38.160799+0000","last_peered":"2024-09-12T02:16:38.160799+0000","last_clean":"2024-09-12T02:16:38.160799+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:16:38.160799+0000","last_undegraded":"2024-09-12T02:16:38.160799+0000","last_fullsized":"2024-09-12T02:16:38.160799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:30.813683+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:17:30.813683+0000","last_peered":"2024-09-12T02:17:30.813683+0000","last_clean":"2024-09-12T02:17:30.813683+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:17:30.813683+0000","last_undegraded":"2024-09-12T02:17:30.813683+0000","last_fullsized":"2024-09-12T02:17:30.813683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137650+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:16:38.137650+0000","last_peered":"2024-09-12T02:16:38.137650+0000","last_clean":"2024-09-12T02:16:38.137650+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:16:38.137650+0000","last_undegraded":"2024-09-12T02:16:38.137650+0000","last_fullsized":"2024-09-12T02:16:38.137650+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.142089+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:16:38.142089+0000","last_peered":"2024-09-12T02:16:38.142089+0000","last_clean":"2024-09-12T02:16:38.142089+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:16:38.142089+0000","last_undegraded":"2024-09-12T02:16:38.142089+0000","last_fullsized":"2024-09-12T02:16:38.142089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739199+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:16:39.739199+0000","last_peered":"2024-09-12T02:16:39.739199+0000","last_clean":"2024-09-12T02:16:39.739199+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:16:39.739199+0000","last_undegraded":"2024-09-12T02:16:39.739199+0000","last_fullsized":"2024-09-12T02:16:39.739199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159706+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:16:38.159706+0000","last_peered":"2024-09-12T02:16:38.159706+0000","last_clean":"2024-09-12T02:16:38.159706+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:16:38.159706+0000","last_undegraded":"2024-09-12T02:16:38.159706+0000","last_fullsized":"2024-09-12T02:16:38.159706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138152+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:16:38.138152+0000","last_peered":"2024-09-12T02:16:38.138152+0000","last_clean":"2024-09-12T02:16:38.138152+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:16:38.138152+0000","last_undegraded":"2024-09-12T02:16:38.138152+0000","last_fullsized":"2024-09-12T02:16:38.138152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138164+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:16:38.138164+0000","last_peered":"2024-09-12T02:16:38.138164+0000","last_clean":"2024-09-12T02:16:38.138164+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:16:38.138164+0000","last_undegraded":"2024-09-12T02:16:38.138164+0000","last_fullsized":"2024-09-12T02:16:38.138164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:29.812695+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:17:29.812695+0000","last_peered":"2024-09-12T02:17:29.812695+0000","last_clean":"2024-09-12T02:17:29.812695+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:17:29.812695+0000","last_undegraded":"2024-09-12T02:17:29.812695+0000","last_fullsized":"2024-09-12T02:17:29.812695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138219+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:16:38.138219+0000","last_peered":"2024-09-12T02:16:38.138219+0000","last_clean":"2024-09-12T02:16:38.138219+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:16:38.138219+0000","last_undegraded":"2024-09-12T02:16:38.138219+0000","last_fullsized":"2024-09-12T02:16:38.138219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":337,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137830+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137830+0000","last_peered":"2024-09-12T02:16:38.137830+0000","last_clean":"2024-09-12T02:16:38.137830+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137830+0000","last_undegraded":"2024-09-12T02:16:38.137830+0000","last_fullsized":"2024-09-12T02:16:38.137830+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138042+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:16:38.138042+0000","last_peered":"2024-09-12T02:16:38.138042+0000","last_clean":"2024-09-12T02:16:38.138042+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:16:38.138042+0000","last_undegraded":"2024-09-12T02:16:38.138042+0000","last_fullsized":"2024-09-12T02:16:38.138042+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157616+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:16:38.157616+0000","last_peered":"2024-09-12T02:16:38.157616+0000","last_clean":"2024-09-12T02:16:38.157616+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:16:38.157616+0000","last_undegraded":"2024-09-12T02:16:38.157616+0000","last_fullsized":"2024-09-12T02:16:38.157616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:31.814402+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:17:31.814402+0000","last_peered":"2024-09-12T02:17:31.814402+0000","last_clean":"2024-09-12T02:17:31.814402+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:17:31.814402+0000","last_undegraded":"2024-09-12T02:17:31.814402+0000","last_fullsized":"2024-09-12T02:17:31.814402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140764+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:16:38.140764+0000","last_peered":"2024-09-12T02:16:38.140764+0000","last_clean":"2024-09-12T02:16:38.140764+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:16:38.140764+0000","last_undegraded":"2024-09-12T02:16:38.140764+0000","last_fullsized":"2024-09-12T02:16:38.140764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'6","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:31.815030+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:17:31.815030+0000","last_peered":"2024-09-12T02:17:31.815030+0000","last_clean":"2024-09-12T02:17:31.815030+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:17:31.815030+0000","last_undegraded":"2024-09-12T02:17:31.815030+0000","last_fullsized":"2024-09-12T02:17:31.815030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156353+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:16:38.156353+0000","last_peered":"2024-09-12T02:16:38.156353+0000","last_clean":"2024-09-12T02:16:38.156353+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:16:38.156353+0000","last_undegraded":"2024-09-12T02:16:38.156353+0000","last_fullsized":"2024-09-12T02:16:38.156353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156722+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:16:38.156722+0000","last_peered":"2024-09-12T02:16:38.156722+0000","last_clean":"2024-09-12T02:16:38.156722+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:16:38.156722+0000","last_undegraded":"2024-09-12T02:16:38.156722+0000","last_fullsized":"2024-09-12T02:16:38.156722+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.156574+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:16:38.156574+0000","last_peered":"2024-09-12T02:16:38.156574+0000","last_clean":"2024-09-12T02:16:38.156574+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:16:38.156574+0000","last_undegraded":"2024-09-12T02:16:38.156574+0000","last_fullsized":"2024-09-12T02:16:38.156574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:55.125142+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:55.125142+0000","last_peered":"2024-09-12T02:16:55.125142+0000","last_clean":"2024-09-12T02:16:55.125142+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:55.125142+0000","last_undegraded":"2024-09-12T02:16:55.125142+0000","last_fullsized":"2024-09-12T02:16:55.125142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138258+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:16:38.138258+0000","last_peered":"2024-09-12T02:16:38.138258+0000","last_clean":"2024-09-12T02:16:38.138258+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:16:38.138258+0000","last_undegraded":"2024-09-12T02:16:38.138258+0000","last_fullsized":"2024-09-12T02:16:38.138258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.161245+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:16:38.161245+0000","last_peered":"2024-09-12T02:16:38.161245+0000","last_clean":"2024-09-12T02:16:38.161245+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:16:38.161245+0000","last_undegraded":"2024-09-12T02:16:38.161245+0000","last_fullsized":"2024-09-12T02:16:38.161245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.139845+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:16:38.139845+0000","last_peered":"2024-09-12T02:16:38.139845+0000","last_clean":"2024-09-12T02:16:38.139845+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:16:38.139845+0000","last_undegraded":"2024-09-12T02:16:38.139845+0000","last_fullsized":"2024-09-12T02:16:38.139845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.159016+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:16:38.159016+0000","last_peered":"2024-09-12T02:16:38.159016+0000","last_clean":"2024-09-12T02:16:38.159016+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:16:38.159016+0000","last_undegraded":"2024-09-12T02:16:38.159016+0000","last_fullsized":"2024-09-12T02:16:38.159016+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158385+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:16:38.158385+0000","last_peered":"2024-09-12T02:16:38.158385+0000","last_clean":"2024-09-12T02:16:38.158385+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:16:38.158385+0000","last_undegraded":"2024-09-12T02:16:38.158385+0000","last_fullsized":"2024-09-12T02:16:38.158385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137813+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:16:38.137813+0000","last_peered":"2024-09-12T02:16:38.137813+0000","last_clean":"2024-09-12T02:16:38.137813+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:16:38.137813+0000","last_undegraded":"2024-09-12T02:16:38.137813+0000","last_fullsized":"2024-09-12T02:16:38.137813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140393+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:16:38.140393+0000","last_peered":"2024-09-12T02:16:38.140393+0000","last_clean":"2024-09-12T02:16:38.140393+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:16:38.140393+0000","last_undegraded":"2024-09-12T02:16:38.140393+0000","last_fullsized":"2024-09-12T02:16:38.140393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135916+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:16:38.135916+0000","last_peered":"2024-09-12T02:16:38.135916+0000","last_clean":"2024-09-12T02:16:38.135916+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:16:38.135916+0000","last_undegraded":"2024-09-12T02:16:38.135916+0000","last_fullsized":"2024-09-12T02:16:38.135916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140265+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:16:38.140265+0000","last_peered":"2024-09-12T02:16:38.140265+0000","last_clean":"2024-09-12T02:16:38.140265+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:16:38.140265+0000","last_undegraded":"2024-09-12T02:16:38.140265+0000","last_fullsized":"2024-09-12T02:16:38.140265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138254+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:16:38.138254+0000","last_peered":"2024-09-12T02:16:38.138254+0000","last_clean":"2024-09-12T02:16:38.138254+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:16:38.138254+0000","last_undegraded":"2024-09-12T02:16:38.138254+0000","last_fullsized":"2024-09-12T02:16:38.138254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:17:28.809894+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:17:28.809894+0000","last_peered":"2024-09-12T02:17:28.809894+0000","last_clean":"2024-09-12T02:17:28.809894+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:17:28.809894+0000","last_undegraded":"2024-09-12T02:17:28.809894+0000","last_fullsized":"2024-09-12T02:17:28.809894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68567,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":554,"num_read_kb":349,"num_write":243,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1091696,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1031743,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438955,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":438423,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":501447,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":501620,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:17:36.753 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:17:36.753 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T02:17:36.753 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T02:17:36.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T02:17:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:37.450 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:37 smithi003 ceph-mon[31476]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:37.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:37 smithi026 ceph-mon[35496]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:37.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:37 smithi092 ceph-mon[36089]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 -- 172.21.15.3:0/700497203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 msgr2=0x7f4698103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 --2- 172.21.15.3:0/700497203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698103110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4688009930 tx=0x7f468802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 -- 172.21.15.3:0/700497203 shutdown_connections 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 --2- 172.21.15.3:0/700497203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981082a0 0x7f469810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 --2- 172.21.15.3:0/700497203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981036e0 0x7f4698107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 --2- 172.21.15.3:0/700497203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698103110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 -- 172.21.15.3:0/700497203 >> 172.21.15.3:0/700497203 conn(0x7f46980fc7d0 msgr2=0x7f46980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 -- 172.21.15.3:0/700497203 shutdown_connections 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.532+0000 7f469ce88640 1 -- 172.21.15.3:0/700497203 wait complete. 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 Processor -- start 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 -- start start 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f46981120a0 con 0x7f4698102d30 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4698111f20 con 0x7f46981036e0 2024-09-12T02:17:37.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469ce88640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4698112220 con 0x7f46981082a0 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53858/0 (socket says 172.21.15.3:53858) 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.533+0000 7f469659c640 1 -- 172.21.15.3:0/2081097843 learned_addr learned my addr 172.21.15.3:0/2081097843 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f4695d9b640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f4696d9d640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 -- 172.21.15.3:0/2081097843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 msgr2=0x7f4698101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 -- 172.21.15.3:0/2081097843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 msgr2=0x7f4698100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 -- 172.21.15.3:0/2081097843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46981a6aa0 con 0x7f4698102d30 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f4696d9d640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:37.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f4695d9b640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469659c640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4688009230 tx=0x7f4688004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468803f030 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46981a6d30 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688004440 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688035070 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.534+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46981a7200 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.535+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4698103110 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.536+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4688042030 con 0x7f4698102d30 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.537+0000 7f46877fe640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 0x7f46800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.537+0000 7f4695d9b640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 0x7f46800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:37.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.537+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f46880ef680 con 0x7f4698102d30 2024-09-12T02:17:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.538+0000 7f4695d9b640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 0x7f46800a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4698102420 tx=0x7f468c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:37.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.541+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46880bc410 con 0x7f4698102d30 2024-09-12T02:17:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.755+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f469810a2b0 con 0x7f4698102d30 2024-09-12T02:17:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.756+0000 7f46877fe640 1 -- 172.21.15.3:0/2081097843 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f46880bbd10 con 0x7f4698102d30 2024-09-12T02:17:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:37.757 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T02:17:37.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 msgr2=0x7f46800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 0x7f46800a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4698102420 tx=0x7f468c005f90 comp rx=0 tx=0).stop 2024-09-12T02:17:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 msgr2=0x7f4698100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4688009230 tx=0x7f4688004040 comp rx=0 tx=0).stop 2024-09-12T02:17:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 shutdown_connections 2024-09-12T02:17:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f46800a2c10 0x7f46800a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46981082a0 0x7f4698101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46981036e0 0x7f4698100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 --2- 172.21.15.3:0/2081097843 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4698102d30 0x7f4698100990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 >> 172.21.15.3:0/2081097843 conn(0x7f46980fc7d0 msgr2=0x7f4698105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 shutdown_connections 2024-09-12T02:17:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:37.758+0000 7f469ce88640 1 -- 172.21.15.3:0/2081097843 wait complete. 2024-09-12T02:17:38.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:38 smithi003 ceph-mon[31476]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:38.417 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2081097843' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:17:38.449 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T02:17:38.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:38 smithi026 ceph-mon[35496]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2081097843' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:17:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:38 smithi092 ceph-mon[36089]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2081097843' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:17:38.711 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:39.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 -- 172.21.15.3:0/2209863178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 msgr2=0x7fb218106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/2209863178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb218106e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb208009930 tx=0x7fb20802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:39.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 -- 172.21.15.3:0/2209863178 shutdown_connections 2024-09-12T02:17:39.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/2209863178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218101420 0x7fb21810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/2209863178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218100a80 0x7fb218100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/2209863178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb218106e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 -- 172.21.15.3:0/2209863178 >> 172.21.15.3:0/2209863178 conn(0x7fb2180fc7a0 msgr2=0x7fb2180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 -- 172.21.15.3:0/2209863178 shutdown_connections 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.204+0000 7fb21cb5b640 1 -- 172.21.15.3:0/2209863178 wait complete. 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 Processor -- start 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 -- start start 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 0x7fb218102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 0x7fb218102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb218110e80 con 0x7fb218106a80 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb218110d00 con 0x7fb218100a80 2024-09-12T02:17:39.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21cb5b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb218111000 con 0x7fb218101420 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb216d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb216d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34118/0 (socket says 172.21.15.3:34118) 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb216d9d640 1 -- 172.21.15.3:0/232828934 learned_addr learned my addr 172.21.15.3:0/232828934 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb21659c640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 0x7fb218102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb20dd9b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 0x7fb218102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.205+0000 7fb216d9d640 1 -- 172.21.15.3:0/232828934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 msgr2=0x7fb218102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb216d9d640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 0x7fb218102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb216d9d640 1 -- 172.21.15.3:0/232828934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 msgr2=0x7fb218102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb216d9d640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 0x7fb218102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb216d9d640 1 -- 172.21.15.3:0/232828934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2181a6950 con 0x7fb218106a80 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb20dd9b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 0x7fb218102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb21659c640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 0x7fb218102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:39.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb216d9d640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb210009fd0 tx=0x7fb21000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210017070 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb210003e50 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2181a6c40 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2100106c0 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.206+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2181a7130 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.207+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb218100ee0 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.208+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb210003970 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.209+0000 7fb20ffff640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 0x7fb2040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.209+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2100cef50 con 0x7fb218106a80 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.209+0000 7fb21659c640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 0x7fb2040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.209+0000 7fb21659c640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 0x7fb2040a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:39.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.212+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb21009bd40 con 0x7fb218106a80 2024-09-12T02:17:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.421+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2181a7360 con 0x7fb218106a80 2024-09-12T02:17:39.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.421+0000 7fb20ffff640 1 -- 172.21.15.3:0/232828934 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7fb210097030 con 0x7fb218106a80 2024-09-12T02:17:39.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:39.422 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:17:39.422 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:17:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.423+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 msgr2=0x7fb2040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.423+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 0x7fb2040a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb208000c00 tx=0x7fb208032c70 comp rx=0 tx=0).stop 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.423+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 msgr2=0x7fb2181030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.423+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb210009fd0 tx=0x7fb21000b260 comp rx=0 tx=0).stop 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 shutdown_connections 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2040a2c10 0x7fb2040a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb218106a80 0x7fb2181030d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb218101420 0x7fb218102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 --2- 172.21.15.3:0/232828934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb218100a80 0x7fb218102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:39.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 >> 172.21.15.3:0/232828934 conn(0x7fb2180fc7a0 msgr2=0x7fb2181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:39.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 shutdown_connections 2024-09-12T02:17:39.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:39.424+0000 7fb21cb5b640 1 -- 172.21.15.3:0/232828934 wait complete. 2024-09-12T02:17:40.087 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:40.350 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:40.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:40 smithi003 ceph-mon[31476]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:40.472 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:40 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/232828934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:40 smithi026 ceph-mon[35496]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:40 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/232828934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:40 smithi092 ceph-mon[36089]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:40 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/232828934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 -- 172.21.15.3:0/130235473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c101de0 msgr2=0x7fb85c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 --2- 172.21.15.3:0/130235473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c101de0 0x7fb85c1021c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009900 tx=0x7fb84c02a410 comp rx=0 tx=0).stop 2024-09-12T02:17:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 -- 172.21.15.3:0/130235473 shutdown_connections 2024-09-12T02:17:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 --2- 172.21.15.3:0/130235473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 0x7fb85c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 --2- 172.21.15.3:0/130235473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c102790 0x7fb85c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 --2- 172.21.15.3:0/130235473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c101de0 0x7fb85c1021c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 -- 172.21.15.3:0/130235473 >> 172.21.15.3:0/130235473 conn(0x7fb85c0fd8c0 msgr2=0x7fb85c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 -- 172.21.15.3:0/130235473 shutdown_connections 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.773+0000 7fb860ff2640 1 -- 172.21.15.3:0/130235473 wait complete. 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 Processor -- start 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 -- start start 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c101de0 0x7fb85c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 0x7fb85c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c111f70 con 0x7fb85c102790 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c111df0 con 0x7fb85c106f10 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb860ff2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c1120f0 con 0x7fb85c101de0 2024-09-12T02:17:40.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34148/0 (socket says 172.21.15.3:34148) 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 -- 172.21.15.3:0/2203698139 learned_addr learned my addr 172.21.15.3:0/2203698139 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb85a59c640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c101de0 0x7fb85c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb85ad9d640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 0x7fb85c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 -- 172.21.15.3:0/2203698139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c101de0 msgr2=0x7fb85c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c101de0 0x7fb85c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 -- 172.21.15.3:0/2203698139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 msgr2=0x7fb85c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 0x7fb85c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.774+0000 7fb859d9b640 1 -- 172.21.15.3:0/2203698139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c19d380 con 0x7fb85c102790 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb859d9b640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c970 tx=0x7fb85000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850007bb0 con 0x7fb85c102790 2024-09-12T02:17:40.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c19d670 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb850007d50 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000f450 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.775+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1a4f50 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.776+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c1021c0 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.777+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb850005020 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.778+0000 7fb84b7fe640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 0x7fb8440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.778+0000 7fb85a59c640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 0x7fb8440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.778+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8500cefc0 con 0x7fb85c102790 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.778+0000 7fb85a59c640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 0x7fb8440a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:40.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.782+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb850014030 con 0x7fb85c102790 2024-09-12T02:17:40.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.993+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb85c109b60 con 0x7fb85c102790 2024-09-12T02:17:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.994+0000 7fb84b7fe640 1 -- 172.21.15.3:0/2203698139 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7fb85009bdb0 con 0x7fb85c102790 2024-09-12T02:17:40.994 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:40.994 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:17:40.994 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 msgr2=0x7fb8440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 0x7fb8440a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 msgr2=0x7fb85c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb85000c970 tx=0x7fb85000ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 shutdown_connections 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8440a2c10 0x7fb8440a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85c106f10 0x7fb85c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb85c102790 0x7fb85c198910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 --2- 172.21.15.3:0/2203698139 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb85c101de0 0x7fb85c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 >> 172.21.15.3:0/2203698139 conn(0x7fb85c0fd8c0 msgr2=0x7fb85c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 shutdown_connections 2024-09-12T02:17:40.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:40.996+0000 7fb860ff2640 1 -- 172.21.15.3:0/2203698139 wait complete. 2024-09-12T02:17:41.320 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:41 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2203698139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:41.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:41 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2203698139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:41.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:41 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2203698139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:41.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:17:41.945 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:42.342 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:42 smithi003 ceph-mon[31476]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:42.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 -- 172.21.15.3:0/1466986066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 msgr2=0x7f60281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 --2- 172.21.15.3:0/1466986066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f60281067e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6024009900 tx=0x7f602402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 -- 172.21.15.3:0/1466986066 shutdown_connections 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 --2- 172.21.15.3:0/1466986066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028106f10 0x7f602810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 --2- 172.21.15.3:0/1466986066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f60281067e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 --2- 172.21.15.3:0/1466986066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028101de0 0x7f60281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 -- 172.21.15.3:0/1466986066 >> 172.21.15.3:0/1466986066 conn(0x7f60280fd8c0 msgr2=0x7f60280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 -- 172.21.15.3:0/1466986066 shutdown_connections 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.423+0000 7f602f8b7640 1 -- 172.21.15.3:0/1466986066 wait complete. 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 Processor -- start 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 -- start start 2024-09-12T02:17:42.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 0x7f6028198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028106f10 0x7f602819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6028111f90 con 0x7f6028102790 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6028111e10 con 0x7f6028101de0 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.424+0000 7f602f8b7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6028112110 con 0x7f6028106f10 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34168/0 (socket says 172.21.15.3:34168) 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 -- 172.21.15.3:0/2095075357 learned_addr learned my addr 172.21.15.3:0/2095075357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602d653640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 0x7f6028198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602de54640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028106f10 0x7f602819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 -- 172.21.15.3:0/2095075357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028106f10 msgr2=0x7f602819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028106f10 0x7f602819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 -- 172.21.15.3:0/2095075357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 msgr2=0x7f6028198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 0x7f6028198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 -- 172.21.15.3:0/2095075357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602819d3c0 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602d653640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 0x7f6028198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602ce52640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6024005ab0 tx=0x7f60240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602403f030 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6024003ce0 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602819d650 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024035070 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.425+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a4f90 con 0x7f6028102790 2024-09-12T02:17:42.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.426+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60281032c0 con 0x7f6028102790 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.428+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6024042030 con 0x7f6028102790 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.429+0000 7f601e7fc640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 0x7f60140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.429+0000 7f602d653640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 0x7f60140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.429+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f60240efe30 con 0x7f6028102790 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.430+0000 7f602d653640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 0x7f60140a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6018009930 tx=0x7f6018008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:42.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.432+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60240bcc20 con 0x7f6028102790 2024-09-12T02:17:42.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:42 smithi026 ceph-mon[35496]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:42.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:42 smithi092 ceph-mon[36089]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:42.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.611+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f602810a0f0 con 0x7f6028102790 2024-09-12T02:17:42.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.612+0000 7f601e7fc640 1 -- 172.21.15.3:0/2095075357 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f60240bc520 con 0x7f6028102790 2024-09-12T02:17:42.613 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:42.614 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:17:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 msgr2=0x7f60140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 0x7f60140a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6018009930 tx=0x7f6018008040 comp rx=0 tx=0).stop 2024-09-12T02:17:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 msgr2=0x7f6028198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6024005ab0 tx=0x7f60240042f0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 shutdown_connections 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60140a2c10 0x7f60140a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6028106f10 0x7f602819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6028102790 0x7f6028198950 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 --2- 172.21.15.3:0/2095075357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6028101de0 0x7f6028198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 >> 172.21.15.3:0/2095075357 conn(0x7f60280fd8c0 msgr2=0x7f60280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:42.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 shutdown_connections 2024-09-12T02:17:42.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:42.614+0000 7f602f8b7640 1 -- 172.21.15.3:0/2095075357 wait complete. 2024-09-12T02:17:43.312 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:17:43.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:43 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2095075357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:43 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2095075357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:43 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2095075357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:17:43.570 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- 172.21.15.3:0/4093596135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8102790 msgr2=0x7f68e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 --2- 172.21.15.3:0/4093596135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8102790 0x7f68e81067e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009900 tx=0x7f68d802a410 comp rx=0 tx=0).stop 2024-09-12T02:17:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- 172.21.15.3:0/4093596135 shutdown_connections 2024-09-12T02:17:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 --2- 172.21.15.3:0/4093596135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8106f10 0x7f68e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 --2- 172.21.15.3:0/4093596135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8102790 0x7f68e81067e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 --2- 172.21.15.3:0/4093596135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8101de0 0x7f68e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- 172.21.15.3:0/4093596135 >> 172.21.15.3:0/4093596135 conn(0x7f68e80fd8c0 msgr2=0x7f68e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- 172.21.15.3:0/4093596135 shutdown_connections 2024-09-12T02:17:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- 172.21.15.3:0/4093596135 wait complete. 2024-09-12T02:17:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 Processor -- start 2024-09-12T02:17:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.075+0000 7f68ef920640 1 -- start start 2024-09-12T02:17:44.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8102790 0x7f68e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 0x7f68e819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8111f70 con 0x7f68e8101de0 2024-09-12T02:17:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8111df0 con 0x7f68e8102790 2024-09-12T02:17:44.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81120f0 con 0x7f68e8106f10 2024-09-12T02:17:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34182/0 (socket says 172.21.15.3:34182) 2024-09-12T02:17:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 -- 172.21.15.3:0/2649523419 learned_addr learned my addr 172.21.15.3:0/2649523419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ecebb640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8102790 0x7f68e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68edebd640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 0x7f68e819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 -- 172.21.15.3:0/2649523419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 msgr2=0x7f68e819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 0x7f68e819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 -- 172.21.15.3:0/2649523419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8102790 msgr2=0x7f68e81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8102790 0x7f68e81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 -- 172.21.15.3:0/2649523419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e819d460 con 0x7f68e8101de0 2024-09-12T02:17:44.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68edebd640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 0x7f68e819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:44.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ed6bc640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68e400c970 tx=0x7f68e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4007bb0 con 0x7f68e8101de0 2024-09-12T02:17:44.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e4007d50 con 0x7f68e8101de0 2024-09-12T02:17:44.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e400f450 con 0x7f68e8101de0 2024-09-12T02:17:44.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.076+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e819d750 con 0x7f68e8101de0 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.077+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e81a5030 con 0x7f68e8101de0 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.077+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68e81021c0 con 0x7f68e8101de0 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.079+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f68e4004fe0 con 0x7f68e8101de0 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.079+0000 7f68de7fc640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 0x7f68d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.079+0000 7f68ecebb640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 0x7f68d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:44.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.079+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f68e40cf330 con 0x7f68e8101de0 2024-09-12T02:17:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.080+0000 7f68ecebb640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 0x7f68d40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009230 tx=0x7f68d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:44.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.083+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68e4014030 con 0x7f68e8101de0 2024-09-12T02:17:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.296+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68e8199810 con 0x7f68e8101de0 2024-09-12T02:17:44.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.297+0000 7f68de7fc640 1 -- 172.21.15.3:0/2649523419 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7f68e8199810 con 0x7f68e8101de0 2024-09-12T02:17:44.297 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:17:44.297 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:17:44.297 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 msgr2=0x7f68d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 0x7f68d40a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009230 tx=0x7f68d80058d0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 msgr2=0x7f68e81984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68e400c970 tx=0x7f68e400ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 shutdown_connections 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68d40a2c90 0x7f68d40a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e8106f10 0x7f68e819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68e8102790 0x7f68e81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 --2- 172.21.15.3:0/2649523419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68e8101de0 0x7f68e81984b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:44.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 >> 172.21.15.3:0/2649523419 conn(0x7f68e80fd8c0 msgr2=0x7f68e80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 shutdown_connections 2024-09-12T02:17:44.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:44.299+0000 7f68ef920640 1 -- 172.21.15.3:0/2649523419 wait complete. 2024-09-12T02:17:44.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:44 smithi026 ceph-mon[35496]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:44.566 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:44 smithi092 ceph-mon[36089]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:44.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:44 smithi003 ceph-mon[31476]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:44.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:16:29.718750+0000', 'modified': '2024-09-12T02:16:40.792890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24508, 'mds_1': 24493, 'mds_2': 24487, 'mds_3': 24482, 'mds_4': 14619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/775355044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 775355044}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 775355044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.3:6841/3157578506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 3157578506}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 3157578506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6833/1804609961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1804609961}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 1804609961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1866420590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1866420590}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1866420590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/3877701831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3877701831}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3877701831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6837/2891911183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 2891911183}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2891911183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6839/1997151172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 1997151172}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 1997151172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6835/1180218932', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1180218932}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1180218932}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6837/487477954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 487477954}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 487477954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/1571545450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1571545450}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1571545450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [14619, 24482, 24508, 24487, 24493]} 2024-09-12T02:17:44.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:17:44.971 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-12T02:17:45.230 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:45.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:45 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2649523419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:45.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:45 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2649523419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:45.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:45 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2649523419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 -- 172.21.15.3:0/3702480711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8106f50 msgr2=0x7f0cd810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 --2- 172.21.15.3:0/3702480711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8106f50 0x7f0cd810eaf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009cd0 tx=0x7f0cd002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 -- 172.21.15.3:0/3702480711 shutdown_connections 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 --2- 172.21.15.3:0/3702480711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8106f50 0x7f0cd810eaf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 --2- 172.21.15.3:0/3702480711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 --2- 172.21.15.3:0/3702480711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd81002f0 0x7f0cd81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 -- 172.21.15.3:0/3702480711 >> 172.21.15.3:0/3702480711 conn(0x7f0cd8077210 msgr2=0x7f0cd80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 -- 172.21.15.3:0/3702480711 shutdown_connections 2024-09-12T02:17:45.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.718+0000 7f0cdff70640 1 -- 172.21.15.3:0/3702480711 wait complete. 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 Processor -- start 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 -- start start 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 0x7f0cd8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 0x7f0cd819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8111fa0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8111e20 con 0x7f0cd8106f50 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdff70640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd8112120 con 0x7f0cd81002f0 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34206/0 (socket says 172.21.15.3:34206) 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.719+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2768610522 learned_addr learned my addr 172.21.15.3:0/2768610522 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:45.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 0x7f0cd8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 0x7f0cd819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2768610522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 msgr2=0x7f0cd8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 0x7f0cd8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2768610522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 msgr2=0x7f0cd819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 0x7f0cd819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2768610522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd819d3f0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 0x7f0cd8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 0x7f0cd819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c970 tx=0x7f0cc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cceffd640 1 -- 172.21.15.3:0/2768610522 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8007bb0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd819d6e0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cceffd640 1 -- 172.21.15.3:0/2768610522 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc8007d50 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cceffd640 1 -- 172.21.15.3:0/2768610522 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800f450 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.720+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a4fc0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.723+0000 7f0cceffd640 1 -- 172.21.15.3:0/2768610522 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0cc8004f90 con 0x7f0cd8100ca0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.723+0000 7f0ce2349740 1 Processor -- start 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 -- start start 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55840a823090 con 0x55840a8c7bb0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55840a7caa70 con 0x55840a8bc9d0 2024-09-12T02:17:45.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0ce2349740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55840a81f050 con 0x55840a8bd2f0 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cddd0c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cde50d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cddd0c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:35052/0 (socket says 172.21.15.3:35052) 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cde50d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:42060/0 (socket says 172.21.15.3:42060) 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cddd0c640 1 -- 172.21.15.3:0/607451272 learned_addr learned my addr 172.21.15.3:0/607451272 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.724+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 -- 172.21.15.3:0/607451272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 msgr2=0x55840a8bcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8bcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 -- 172.21.15.3:0/607451272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 msgr2=0x55840a8c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 -- 172.21.15.3:0/607451272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55840a7d7130 con 0x55840a8bd2f0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cceffd640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400bb70 tx=0x7f0cd400d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513df4252fd8dba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cc3fff640 1 -- 172.21.15.3:0/607451272 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017620 con 0x55840a8bd2f0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cc3fff640 1 -- 172.21.15.3:0/607451272 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4017ca0 con 0x55840a8bd2f0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cceffd640 1 -- 172.21.15.3:0/2768610522 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0cc80ceff0 con 0x7f0cd8100ca0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.725+0000 7f0cc3fff640 1 -- 172.21.15.3:0/607451272 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400fd10 con 0x55840a8bd2f0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- 172.21.15.3:0/607451272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 msgr2=0x55840a8c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009430 tx=0x7f0cd003f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400bb70 tx=0x7f0cd400d710 comp rx=0 tx=0).stop 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- 172.21.15.3:0/607451272 shutdown_connections 2024-09-12T02:17:45.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c7670 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 --2- 172.21.15.3:0/607451272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8bcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- 172.21.15.3:0/607451272 >> 172.21.15.3:0/607451272 conn(0x55840a8b8f60 msgr2=0x55840a8b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- 172.21.15.3:0/607451272 shutdown_connections 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- 172.21.15.3:0/607451272 wait complete. 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.726+0000 7f0ce2349740 1 -- start start 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 Processor -- start 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55840a8ca780 con 0x55840a8bd2f0 2024-09-12T02:17:45.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55840a7ddf20 con 0x55840a8c7bb0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55840a8ca900 con 0x55840a8bc9d0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0ce2349740 10 client.0 _ll_register_callbacks cb 0x55840a7a7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34218/0 (socket says 172.21.15.3:34218) 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 -- 172.21.15.3:0/1114261107 learned_addr learned my addr 172.21.15.3:0/1114261107 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cde50d640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 -- 172.21.15.3:0/1114261107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 msgr2=0x55840a8c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 -- 172.21.15.3:0/1114261107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 msgr2=0x55840a8c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.727+0000 7f0cdd50b640 1 -- 172.21.15.3:0/1114261107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55840a5942b0 con 0x55840a8bd2f0 2024-09-12T02:17:45.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8007590 tx=0x7f0cc800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfc0979e3b4ee25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc801d040 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0cc80ccba0 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd4017ca0 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cc809b1e0 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 10 client.14979 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55840a8d5f70 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4017ca0 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0cc80d1110 con 0x55840a8bd2f0 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 10 client.14979 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 10 client.14979 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:17:45.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 20 client.14979 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.728+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a8d8570 0x55840a8d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.729+0000 7f0ce2349740 4 client.14979 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.729+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55840a830600 con 0x55840a8d8570 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.729+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a8d8570 0x55840a8d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.729+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a8d8570 0x55840a8d99e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4903dba21482b452 server_cookie=89de2a7f89f9e051 in_seq=0 out_seq=0 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.729+0000 7f0cc17fa640 10 client.14979 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0cc17fa640 1 -- 172.21.15.3:0/1114261107 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55840a830600 con 0x55840a8d8570 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0cc17fa640 10 client.14979 handle_command_reply: tid=0 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 client.14979 shutdown 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 msgr2=0x55840a8c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8007590 tx=0x7f0cc800a560 comp rx=0 tx=0).stop 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 shutdown_connections 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8c4690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a8d8570 0x55840a8d99e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 --2- 172.21.15.3:0/1114261107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 >> 172.21.15.3:0/1114261107 conn(0x55840a8b8f60 msgr2=0x55840a8bec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0cc17fa640 0 client.14979 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 shutdown_connections 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 1 -- 172.21.15.3:0/1114261107 wait complete. 2024-09-12T02:17:45.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.730+0000 7f0ce2349740 20 client.14979 trim_cache size 0 max 16384 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 Processor -- start 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 -- start start 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d7d20 con 0x55840a8c7bb0 2024-09-12T02:17:45.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d83f0 con 0x55840a8bc9d0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.748+0000 7f0ce2349740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d8ac0 con 0x55840a8bd2f0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34220/0 (socket says 172.21.15.3:34220) 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 -- 172.21.15.3:0/2546174483 learned_addr learned my addr 172.21.15.3:0/2546174483 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 -- 172.21.15.3:0/2546174483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 msgr2=0x55840a8c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 -- 172.21.15.3:0/2546174483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 msgr2=0x55840a8d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 -- 172.21.15.3:0/2546174483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55840a8bd120 con 0x55840a8c7bb0 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:45.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0031c40 tx=0x7f0cd0043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f120202d23c41d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cc37fe640 1 -- 172.21.15.3:0/2546174483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0058070 con 0x55840a8c7bb0 2024-09-12T02:17:45.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cc37fe640 1 -- 172.21.15.3:0/2546174483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd0036040 con 0x55840a8c7bb0 2024-09-12T02:17:45.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.749+0000 7f0cc37fe640 1 -- 172.21.15.3:0/2546174483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0030800 con 0x55840a8c7bb0 2024-09-12T02:17:45.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- 172.21.15.3:0/2546174483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 msgr2=0x55840a8c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0031c40 tx=0x7f0cd0043000 comp rx=0 tx=0).stop 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- 172.21.15.3:0/2546174483 shutdown_connections 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8c7bb0 0x55840a8c4b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bd2f0 0x55840a8c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2546174483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8bc9d0 0x55840a8d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- 172.21.15.3:0/2546174483 >> 172.21.15.3:0/2546174483 conn(0x55840a8b8f60 msgr2=0x55840a8b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- 172.21.15.3:0/2546174483 shutdown_connections 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- 172.21.15.3:0/2546174483 wait complete. 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 -- start start 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.750+0000 7f0ce2349740 1 Processor -- start 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d9860 con 0x55840a8bd2f0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d9190 con 0x55840a8c7bb0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55840a8d9f30 con 0x55840a8bc9d0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0ce2349740 10 client.0 _ll_register_callbacks cb 0x7f0cc80058a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34226/0 (socket says 172.21.15.3:34226) 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2749484760 learned_addr learned my addr 172.21.15.3:0/2749484760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cddd0c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cde50d640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2749484760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 msgr2=0x55840a8e1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8e1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2749484760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 msgr2=0x55840a8db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 -- 172.21.15.3:0/2749484760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55840a8dba30 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cdd50b640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c940 tx=0x7f0cc809b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9029e933205ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.751+0000 7f0cc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc801d070 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd0036040 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0cc80d1110 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cc809b000 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 10 client.14985 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55840a8e0510 con 0x55840a8bd2f0 2024-09-12T02:17:45.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd0036040 con 0x55840a8bd2f0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0cc80a3ed0 con 0x55840a8bd2f0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 10 client.14985 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 10 client.14985 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 20 client.14985 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a9472c0 0x55840a945fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 4 client.14985 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "status"}) -- 0x55840a82ab10 con 0x55840a9472c0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.752+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a9472c0 0x55840a945fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0cddd0c640 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a9472c0 0x55840a945fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79e2012361eb4407 server_cookie=cdb382b451c9892a in_seq=0 out_seq=0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0cc17fa640 10 client.14985 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0cc17fa640 1 -- 172.21.15.3:0/2749484760 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55840a82ab10 con 0x55840a9472c0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0cc17fa640 10 client.14985 handle_command_reply: tid=0 2024-09-12T02:17:45.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 client.14985 shutdown 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 msgr2=0x55840a8e2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c940 tx=0x7f0cc809b5a0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 shutdown_connections 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55840a8c7bb0 0x55840a8db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55840a8bd2f0 0x55840a8e2430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55840a9472c0 0x55840a945fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 --2- 172.21.15.3:0/2749484760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55840a8bc9d0 0x55840a8e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0cc17fa640 0 client.14985 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 >> 172.21.15.3:0/2749484760 conn(0x55840a8b8f60 msgr2=0x55840a8c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 shutdown_connections 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 1 -- 172.21.15.3:0/2749484760 wait complete. 2024-09-12T02:17:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.753+0000 7f0ce2349740 20 client.14985 trim_cache size 0 max 16384 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14619, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 66.016611280000006, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:17:45.760 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 89.441783747000002 2024-09-12T02:17:45.761 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:17:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 msgr2=0x7f0cc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009430 tx=0x7f0cd003f040 comp rx=0 tx=0).stop 2024-09-12T02:17:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 msgr2=0x7f0cd8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800c970 tx=0x7f0cc800ce30 comp rx=0 tx=0).stop 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 shutdown_connections 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd8106f50 0x7f0cd819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cd8100ca0 0x7f0cd8198980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 --2- 172.21.15.3:0/2768610522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd81002f0 0x7f0cd8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 >> 172.21.15.3:0/2768610522 conn(0x7f0cd8077210 msgr2=0x7f0cd8105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 shutdown_connections 2024-09-12T02:17:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:45.754+0000 7f0cdff70640 1 -- 172.21.15.3:0/2768610522 wait complete. 2024-09-12T02:17:46.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:46 smithi003 ceph-mon[31476]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:46.440 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:17:46.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:46 smithi026 ceph-mon[35496]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:46.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:46 smithi092 ceph-mon[36089]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:46.698 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.171+0000 7f85a1f3c640 1 -- 172.21.15.3:0/1138220441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c103660 msgr2=0x7f859c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.171+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/1138220441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c103660 0x7f859c10bb50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8590009900 tx=0x7f859002a480 comp rx=0 tx=0).stop 2024-09-12T02:17:47.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 -- 172.21.15.3:0/1138220441 shutdown_connections 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/1138220441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c10c170 0x7f859c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/1138220441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c103660 0x7f859c10bb50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/1138220441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c102d40 0x7f859c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 -- 172.21.15.3:0/1138220441 >> 172.21.15.3:0/1138220441 conn(0x7f859c0fc7a0 msgr2=0x7f859c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 -- 172.21.15.3:0/1138220441 shutdown_connections 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 -- 172.21.15.3:0/1138220441 wait complete. 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 Processor -- start 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.172+0000 7f85a1f3c640 1 -- start start 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c103660 0x7f859c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c10c170 0x7f859c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f859c111f70 con 0x7f859c102d40 2024-09-12T02:17:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f859c111df0 con 0x7f859c10c170 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f85a1f3c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f859c1120f0 con 0x7f859c103660 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34254/0 (socket says 172.21.15.3:34254) 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 -- 172.21.15.3:0/3161747262 learned_addr learned my addr 172.21.15.3:0/3161747262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859affd640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c103660 0x7f859c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859bfff640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c10c170 0x7f859c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 -- 172.21.15.3:0/3161747262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c103660 msgr2=0x7f859c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c103660 0x7f859c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 -- 172.21.15.3:0/3161747262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c10c170 msgr2=0x7f859c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c10c170 0x7f859c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 -- 172.21.15.3:0/3161747262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1a6930 con 0x7f859c102d40 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c890 tx=0x7f858c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.173+0000 7f8598ff9640 1 -- 172.21.15.3:0/3161747262 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c004220 con 0x7f859c102d40 2024-09-12T02:17:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.174+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c1a6c20 con 0x7f859c102d40 2024-09-12T02:17:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.174+0000 7f8598ff9640 1 -- 172.21.15.3:0/3161747262 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f858c0043c0 con 0x7f859c102d40 2024-09-12T02:17:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.174+0000 7f8598ff9640 1 -- 172.21.15.3:0/3161747262 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858c004a00 con 0x7f859c102d40 2024-09-12T02:17:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.174+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1a7130 con 0x7f859c102d40 2024-09-12T02:17:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f8598ff9640 1 -- 172.21.15.3:0/3161747262 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f858c0079b0 con 0x7f859c102d40 2024-09-12T02:17:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 Processor -- start 2024-09-12T02:17:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 -- start start 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001ca02f0 0x55f001caa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cacfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f001c06090 con 0x55f001c9f9d0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f001bada70 con 0x55f001caabb0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f85a4315740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f001c02050 con 0x55f001ca02f0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f859bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f859bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34256/0 (socket says 172.21.15.3:34256) 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f859bfff640 1 -- 172.21.15.3:0/3790135149 learned_addr learned my addr 172.21.15.3:0/3790135149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001ca02f0 0x55f001caa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f8598ff9640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 0x7f85840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.177+0000 7f859affd640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cacfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 -- 172.21.15.3:0/3790135149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001ca02f0 msgr2=0x55f001caa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001ca02f0 0x55f001caa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 -- 172.21.15.3:0/3790135149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 msgr2=0x55f001cacfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cacfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 -- 172.21.15.3:0/3790135149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f001bba130 con 0x55f001c9f9d0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859affd640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 0x7f85840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f8598ff9640 1 -- 172.21.15.3:0/3161747262 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f858c0ceee0 con 0x7f859c102d40 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f859bfff640 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8594009fc0 tx=0x7f8594010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4aaec69b07bec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f8581ffb640 1 -- 172.21.15.3:0/3790135149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859401c900 con 0x55f001c9f9d0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f8581ffb640 1 -- 172.21.15.3:0/3790135149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859401b3d0 con 0x55f001c9f9d0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.178+0000 7f8581ffb640 1 -- 172.21.15.3:0/3790135149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594025620 con 0x55f001c9f9d0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- 172.21.15.3:0/3790135149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 msgr2=0x55f001c9fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f859bfff640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 0x7f85840a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8594020000 tx=0x7f859401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8594009fc0 tx=0x7f8594010800 comp rx=0 tx=0).stop 2024-09-12T02:17:47.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- 172.21.15.3:0/3790135149 shutdown_connections 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cacfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001ca02f0 0x55f001caa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 --2- 172.21.15.3:0/3790135149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001c9f9d0 0x55f001c9fdb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- 172.21.15.3:0/3790135149 >> 172.21.15.3:0/3790135149 conn(0x55f001c9bf60 msgr2=0x55f001c9c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- 172.21.15.3:0/3790135149 shutdown_connections 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- 172.21.15.3:0/3790135149 wait complete. 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 -- start start 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.179+0000 7f85a4315740 1 Processor -- start 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001caa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001ca7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f001cad780 con 0x55f001ca02f0 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f001bc0f20 con 0x55f001c9f9d0 2024-09-12T02:17:47.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f001cad900 con 0x55f001caabb0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f85a4315740 10 client.0 _ll_register_callbacks cb 0x55f001b8a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34272/0 (socket says 172.21.15.3:34272) 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 -- 172.21.15.3:0/3007430554 learned_addr learned my addr 172.21.15.3:0/3007430554 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859bfff640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001ca7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001caa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 -- 172.21.15.3:0/3007430554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 msgr2=0x55f001ca7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001ca7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 -- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 msgr2=0x55f001caa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001caa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859affd640 1 -- 172.21.15.3:0/3007430554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0019772b0 con 0x55f001ca02f0 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001caa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:47.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.180+0000 7f859bfff640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001ca7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:17:47.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f859affd640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8590009900 tx=0x7f8590031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b9b1dd4db427a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:47.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590040070 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8590004450 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859401b3d0 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8590035040 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f85a4315740 10 client.14997 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f001cb8fd0 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859401b3d0 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.181+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f85900035f0 con 0x55f001ca02f0 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 10 client.14997 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 20 client.14997 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001cbb610 0x55f001cbca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 4 client.14997 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:47.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f001c13600 con 0x55f001cbb610 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001cbb610 0x55f001cbca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001cbb610 0x55f001cbca80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3dc9ad69900a2152 server_cookie=3d6bce34ebaf7c9f in_seq=0 out_seq=0 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.182+0000 7f857affd640 10 client.14997 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f857affd640 1 -- 172.21.15.3:0/3007430554 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f001c13600 con 0x55f001cbb610 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f857affd640 10 client.14997 handle_command_reply: tid=0 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 client.14997 shutdown 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 msgr2=0x55f001ca7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8590009900 tx=0x7f8590031920 comp rx=0 tx=0).stop 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 shutdown_connections 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001ca7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca02f0 0x55f001ca7700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.183+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001cbb610 0x55f001cbca80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f85a4315740 1 --2- 172.21.15.3:0/3007430554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001caa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 >> 172.21.15.3:0/3007430554 conn(0x55f001c9bf60 msgr2=0x55f001ca1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f857affd640 0 client.14997 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:17:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 shutdown_connections 2024-09-12T02:17:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f85a4315740 1 -- 172.21.15.3:0/3007430554 wait complete. 2024-09-12T02:17:47.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.184+0000 7f85a4315740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 Processor -- start 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 -- start start 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001c9f9d0 0x55f001cbb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cc4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbadc0 con 0x55f001ca66e0 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbb490 con 0x55f001caabb0 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f85a4315740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbbb60 con 0x55f001c9f9d0 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34274/0 (socket says 172.21.15.3:34274) 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 -- 172.21.15.3:0/907738616 learned_addr learned my addr 172.21.15.3:0/907738616 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:47.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859b7fe640 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001c9f9d0 0x55f001cbb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859affd640 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cc4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 -- 172.21.15.3:0/907738616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001c9f9d0 msgr2=0x55f001cbb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001c9f9d0 0x55f001cbb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 -- 172.21.15.3:0/907738616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 msgr2=0x55f001cc4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cc4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 -- 172.21.15.3:0/907738616 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f001ca0120 con 0x55f001ca66e0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.202+0000 7f859bfff640 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8594005bd0 tx=0x7f8594027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477144851ccf2d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85817fa640 1 -- 172.21.15.3:0/907738616 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594034020 con 0x55f001ca66e0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85817fa640 1 -- 172.21.15.3:0/907738616 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859401bd80 con 0x55f001ca66e0 2024-09-12T02:17:47.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85817fa640 1 -- 172.21.15.3:0/907738616 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859401b6c0 con 0x55f001ca66e0 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 -- 172.21.15.3:0/907738616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 msgr2=0x55f001cc52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8594005bd0 tx=0x7f8594027000 comp rx=0 tx=0).stop 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 -- 172.21.15.3:0/907738616 shutdown_connections 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc52a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001caabb0 0x55f001cc4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 --2- 172.21.15.3:0/907738616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001c9f9d0 0x55f001cbb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 -- 172.21.15.3:0/907738616 >> 172.21.15.3:0/907738616 conn(0x55f001c9bf60 msgr2=0x55f001ca5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 -- 172.21.15.3:0/907738616 shutdown_connections 2024-09-12T02:17:47.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.203+0000 7f85a4315740 1 -- 172.21.15.3:0/907738616 wait complete. 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 -- start start 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 Processor -- start 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001cbfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001d23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbc900 con 0x55f001ca66e0 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbc230 con 0x55f001c9f9d0 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f001cbcfd0 con 0x55f001caabb0 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f85a4315740 10 client.0 _ll_register_callbacks cb 0x7f8590009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f859affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f859affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34280/0 (socket says 172.21.15.3:34280) 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f859affd640 1 -- 172.21.15.3:0/3507972386 learned_addr learned my addr 172.21.15.3:0/3507972386 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:47.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f859bfff640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001d23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.204+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001cbfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 -- 172.21.15.3:0/3507972386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 msgr2=0x55f001d23d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001d23d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 -- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 msgr2=0x55f001cbfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001cbfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 -- 172.21.15.3:0/3507972386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f001cc0bb0 con 0x55f001ca66e0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f859affd640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8590003bb0 tx=0x7f8590005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829b354bad255bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590040040 con 0x55f001ca66e0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8590005030 con 0x55f001ca66e0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859401bd80 con 0x55f001ca66e0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f859005e050 con 0x55f001ca66e0 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f85a4315740 10 client.15003 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:47.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f001d25320 con 0x55f001ca66e0 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859401bd80 con 0x55f001ca66e0 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.205+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8590004b40 con 0x55f001ca66e0 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 10 client.15003 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 10 client.15003 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 20 client.15003 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001d29f10 0x55f001d28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 4 client.15003 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "status"}) -- 0x55f001c0db10 con 0x55f001d29f10 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001d29f10 0x55f001d28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f857affd640 10 client.15003 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f859b7fe640 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001d29f10 0x55f001d28c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15460696fb9d841f server_cookie=15f1117fb3a3e729 in_seq=0 out_seq=0 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f857affd640 1 -- 172.21.15.3:0/3507972386 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f001c0db10 con 0x55f001d29f10 2024-09-12T02:17:47.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f857affd640 10 client.15003 handle_command_reply: tid=0 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.206+0000 7f85a4315740 1 client.15003 shutdown 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 msgr2=0x55f001cc0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8590003bb0 tx=0x7f8590005e00 comp rx=0 tx=0).stop 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 shutdown_connections 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f001caabb0 0x55f001d23d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f001ca66e0 0x55f001cc0320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55f001d29f10 0x55f001d28c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 --2- 172.21.15.3:0/3507972386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f001c9f9d0 0x55f001cbfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 >> 172.21.15.3:0/3507972386 conn(0x55f001c9bf60 msgr2=0x55f001ca6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f857affd640 0 client.15003 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 shutdown_connections 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 1 -- 172.21.15.3:0/3507972386 wait complete. 2024-09-12T02:17:47.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a4315740 20 client.15003 trim_cache size 0 max 16384 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24482, 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 68.045484294000005, 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:17:47.213 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:17:47.214 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:17:47.214 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 88.027622938999997 2024-09-12T02:17:47.214 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:17:47.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 msgr2=0x7f85840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 0x7f85840a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8594020000 tx=0x7f859401d000 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 msgr2=0x7f859c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c890 tx=0x7f858c00cd50 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.207+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 shutdown_connections 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85840a2c10 0x7f85840a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c10c170 0x7f859c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f859c103660 0x7f859c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 --2- 172.21.15.3:0/3161747262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f859c102d40 0x7f859c1009a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 >> 172.21.15.3:0/3161747262 conn(0x7f859c0fc7a0 msgr2=0x7f859c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 shutdown_connections 2024-09-12T02:17:47.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:47.208+0000 7f85a1f3c640 1 -- 172.21.15.3:0/3161747262 wait complete. 2024-09-12T02:17:47.932 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:17:48.187 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:48.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:48 smithi026 ceph-mon[35496]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:48.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:48 smithi003 ceph-mon[31476]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:48.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:48 smithi092 ceph-mon[36089]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:48.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 -- 172.21.15.3:0/524471551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c102d40 msgr2=0x7fdf5c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 --2- 172.21.15.3:0/524471551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c102d40 0x7fdf5c103120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009900 tx=0x7fdf4c02a480 comp rx=0 tx=0).stop 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 -- 172.21.15.3:0/524471551 shutdown_connections 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 --2- 172.21.15.3:0/524471551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c10c170 0x7fdf5c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 --2- 172.21.15.3:0/524471551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c103660 0x7fdf5c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 --2- 172.21.15.3:0/524471551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c102d40 0x7fdf5c103120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 -- 172.21.15.3:0/524471551 >> 172.21.15.3:0/524471551 conn(0x7fdf5c0fc7a0 msgr2=0x7fdf5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 -- 172.21.15.3:0/524471551 shutdown_connections 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.649+0000 7fdf60854640 1 -- 172.21.15.3:0/524471551 wait complete. 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 Processor -- start 2024-09-12T02:17:48.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 -- start start 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c102d40 0x7fdf5c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c103660 0x7fdf5c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c111f70 con 0x7fdf5c10c170 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c111df0 con 0x7fdf5c103660 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf60854640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c1120f0 con 0x7fdf5c102d40 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf5ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf5ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47222/0 (socket says 172.21.15.3:47222) 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf5ad9d640 1 -- 172.21.15.3:0/2520258171 learned_addr learned my addr 172.21.15.3:0/2520258171 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c102d40 0x7fdf5c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.650+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c103660 0x7fdf5c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 -- 172.21.15.3:0/2520258171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c102d40 msgr2=0x7fdf5c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c102d40 0x7fdf5c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 -- 172.21.15.3:0/2520258171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c103660 msgr2=0x7fdf5c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c103660 0x7fdf5c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 -- 172.21.15.3:0/2520258171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c19d420 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf5ad9d640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009fd0 tx=0x7fdf5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf4b7fe640 1 -- 172.21.15.3:0/2520258171 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54017070 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf4b7fe640 1 -- 172.21.15.3:0/2520258171 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54003e50 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c19d710 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf4b7fe640 1 -- 172.21.15.3:0/2520258171 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf540106c0 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.651+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a4ff0 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.653+0000 7fdf4b7fe640 1 -- 172.21.15.3:0/2520258171 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdf54003970 con 0x7fdf5c10c170 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 Processor -- start 2024-09-12T02:17:48.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 -- start start 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 0x564b5bf2adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf35670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b5be91090 con 0x564b5bf2a9d0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b5be38a70 con 0x564b5bf2b2f0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b5be8d050 con 0x564b5bf35bb0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 0x564b5bf2adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 0x564b5bf2adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47228/0 (socket says 172.21.15.3:47228) 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf59d9b640 1 -- 172.21.15.3:0/3774103614 learned_addr learned my addr 172.21.15.3:0/3774103614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf4b7fe640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 0x7fdf440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf35670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.654+0000 7fdf5ad9d640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 0x7fdf440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 -- 172.21.15.3:0/3774103614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 msgr2=0x564b5bf37fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf37fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 -- 172.21.15.3:0/3774103614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 msgr2=0x564b5bf2adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 0x564b5bf2adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 -- 172.21.15.3:0/3774103614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5be45130 con 0x564b5bf2b2f0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf4b7fe640 1 -- 172.21.15.3:0/2520258171 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdf540cee30 con 0x7fdf5c10c170 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf35670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c005940 tx=0x7fdf4c005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ca7dee1cf95cb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf5ad9d640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 0x7fdf440a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf54096010 tx=0x7fdf54086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf48ff9640 1 -- 172.21.15.3:0/3774103614 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c040070 con 0x564b5bf2b2f0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf48ff9640 1 -- 172.21.15.3:0/3774103614 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf4c0318f0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.655+0000 7fdf48ff9640 1 -- 172.21.15.3:0/3774103614 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c035040 con 0x564b5bf2b2f0 2024-09-12T02:17:48.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3774103614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 msgr2=0x564b5bf35670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf35670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c005940 tx=0x7fdf4c005e00 comp rx=0 tx=0).stop 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3774103614 shutdown_connections 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf37fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf35670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3774103614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2a9d0 0x564b5bf2adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3774103614 >> 172.21.15.3:0/3774103614 conn(0x564b5bf26f60 msgr2=0x564b5bf27340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3774103614 shutdown_connections 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3774103614 wait complete. 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 -- start start 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.656+0000 7fdf62c2d740 1 Processor -- start 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2a9d0 0x564b5bf35460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf38780 con 0x564b5bf35bb0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b5be4bf20 con 0x564b5bf2b2f0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf38900 con 0x564b5bf2a9d0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf62c2d740 10 client.0 _ll_register_callbacks cb 0x564b5be15030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47230/0 (socket says 172.21.15.3:47230) 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf59d9b640 1 -- 172.21.15.3:0/15021467 learned_addr learned my addr 172.21.15.3:0/15021467 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52320/0 (socket says 172.21.15.3:52320) 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2a9d0 0x564b5bf35460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 -- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2a9d0 msgr2=0x564b5bf35460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2a9d0 0x564b5bf35460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 -- 172.21.15.3:0/15021467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 msgr2=0x564b5bf32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf32cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 -- 172.21.15.3:0/15021467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5bc022b0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009900 tx=0x7fdf4c005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248ee5c830e46c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.657+0000 7fdf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c040030 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdf4c0316f0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf4c0318f0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf4c035070 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 10 client.24731 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564b5bf44010 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf4c0318f0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf4c057050 con 0x564b5bf2b2f0 2024-09-12T02:17:48.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 10 client.24731 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 10 client.24731 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 20 client.24731 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bf46210 0x564b5bf47680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 4 client.24731 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.658+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b5be9e600 con 0x564b5bf46210 2024-09-12T02:17:48.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.659+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bf46210 0x564b5bf47680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.659+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bf46210 0x564b5bf47680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=716859242dc8ecfa server_cookie=e60ee2e69e6d483f in_seq=0 out_seq=0 2024-09-12T02:17:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.659+0000 7fdf41ffb640 10 client.24731 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:17:48.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf41ffb640 1 -- 172.21.15.3:0/15021467 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b5be9e600 con 0x564b5bf46210 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf41ffb640 10 client.24731 handle_command_reply: tid=0 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 client.24731 shutdown 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 msgr2=0x564b5bf327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009900 tx=0x7fdf4c005250 comp rx=0 tx=0).stop 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 shutdown_connections 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2b2f0 0x564b5bf327b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bf46210 0x564b5bf47680 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/15021467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2a9d0 0x564b5bf35460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 >> 172.21.15.3:0/15021467 conn(0x564b5bf26f60 msgr2=0x564b5bf2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf41ffb640 0 client.24731 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 shutdown_connections 2024-09-12T02:17:48.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 1 -- 172.21.15.3:0/15021467 wait complete. 2024-09-12T02:17:48.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.660+0000 7fdf62c2d740 20 client.24731 trim_cache size 0 max 16384 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.678+0000 7fdf62c2d740 1 Processor -- start 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.678+0000 7fdf62c2d740 1 -- start start 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.678+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf45360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2b2f0 0x564b5bf33f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf459c0 con 0x564b5bf35bb0 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf46090 con 0x564b5bf2a9d0 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf46760 con 0x564b5bf2b2f0 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47238/0 (socket says 172.21.15.3:47238) 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 -- 172.21.15.3:0/3509740863 learned_addr learned my addr 172.21.15.3:0/3509740863 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf45360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2b2f0 0x564b5bf33f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 -- 172.21.15.3:0/3509740863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2b2f0 msgr2=0x564b5bf33f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2b2f0 0x564b5bf33f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 -- 172.21.15.3:0/3509740863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 msgr2=0x564b5bf45360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf45360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 -- 172.21.15.3:0/3509740863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5bf35080 con 0x564b5bf35bb0 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000fb50 tx=0x7fdf50005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ed61b82dd59ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.679+0000 7fdf43fff640 1 -- 172.21.15.3:0/3509740863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50023920 con 0x564b5bf35bb0 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf43fff640 1 -- 172.21.15.3:0/3509740863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf50023ac0 con 0x564b5bf35bb0 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf43fff640 1 -- 172.21.15.3:0/3509740863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500054d0 con 0x564b5bf35bb0 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3509740863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 msgr2=0x564b5bf34440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000fb50 tx=0x7fdf50005f90 comp rx=0 tx=0).stop 2024-09-12T02:17:48.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3509740863 shutdown_connections 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf35bb0 0x564b5bf34440 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf2b2f0 0x564b5bf33f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/3509740863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf45360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3509740863 >> 172.21.15.3:0/3509740863 conn(0x564b5bf26f60 msgr2=0x564b5bf318c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3509740863 shutdown_connections 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- 172.21.15.3:0/3509740863 wait complete. 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.680+0000 7fdf62c2d740 1 -- start start 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 Processor -- start 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf4fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf48e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf47500 con 0x564b5bf2b2f0 2024-09-12T02:17:48.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf46e30 con 0x564b5bf2a9d0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b5bf47bd0 con 0x564b5bf35bb0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf62c2d740 10 client.0 _ll_register_callbacks cb 0x7fdf50009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47254/0 (socket says 172.21.15.3:47254) 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 -- 172.21.15.3:0/2355743033 learned_addr learned my addr 172.21.15.3:0/2355743033 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf4fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf48e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 -- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 msgr2=0x564b5bf48e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf48e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 -- 172.21.15.3:0/2355743033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 msgr2=0x564b5bf4fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf4fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 -- 172.21.15.3:0/2355743033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5bf49530 con 0x564b5bf2b2f0 2024-09-12T02:17:48.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf48e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf5a59c640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x564b5bf33df0 tx=0x7fdf4c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777a7b4f36831648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.681+0000 7fdf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c040050 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdf4c04a040 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf50023ac0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdf4c005d50 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 10 client.15021 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564b5bf4df60 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf50023ac0 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdf4c057040 con 0x564b5bf2b2f0 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 10 client.15021 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:48.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 10 client.15021 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 20 client.15021 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bfb4eb0 0x564b5bfb3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 4 client.15021 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.682+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "status"}) -- 0x564b5be98b10 con 0x564b5bfb4eb0 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bfb4eb0 0x564b5bfb3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf59d9b640 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bfb4eb0 0x564b5bfb3bc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39b437446660c5ca server_cookie=279e342f6870102b in_seq=0 out_seq=0 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf41ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf41ffb640 1 -- 172.21.15.3:0/2355743033 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564b5be98b10 con 0x564b5bfb4eb0 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf41ffb640 10 client.15021 handle_command_reply: tid=0 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 client.15021 shutdown 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 msgr2=0x564b5bf4ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x564b5bf33df0 tx=0x7fdf4c0038a0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 shutdown_connections 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b5bfb4eb0 0x564b5bfb3bc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b5bf35bb0 0x564b5bf48e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b5bf2b2f0 0x564b5bf4ffa0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 --2- 172.21.15.3:0/2355743033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b5bf2a9d0 0x564b5bf4fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf41ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 >> 172.21.15.3:0/2355743033 conn(0x564b5bf26f60 msgr2=0x564b5bf33e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 shutdown_connections 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.683+0000 7fdf62c2d740 1 -- 172.21.15.3:0/2355743033 wait complete. 2024-09-12T02:17:48.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf62c2d740 20 client.15021 trim_cache size 0 max 16384 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24487, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 70.534557468000003, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:17:48.690 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 106.509798688 2024-09-12T02:17:48.691 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:17:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 msgr2=0x7fdf440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 0x7fdf440a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdf54096010 tx=0x7fdf54086040 comp rx=0 tx=0).stop 2024-09-12T02:17:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 msgr2=0x7fdf5c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:48.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009fd0 tx=0x7fdf5400b260 comp rx=0 tx=0).stop 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 shutdown_connections 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdf440a2c90 0x7fdf440a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf5c10c170 0x7fdf5c19cd40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf5c103660 0x7fdf5c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 --2- 172.21.15.3:0/2520258171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf5c102d40 0x7fdf5c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 >> 172.21.15.3:0/2520258171 conn(0x7fdf5c0fc7a0 msgr2=0x7fdf5c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 shutdown_connections 2024-09-12T02:17:48.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:48.684+0000 7fdf60854640 1 -- 172.21.15.3:0/2520258171 wait complete. 2024-09-12T02:17:49.374 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:17:49.629 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 -- 172.21.15.3:0/101875814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578100a80 msgr2=0x7f6578100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 --2- 172.21.15.3:0/101875814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578100a80 0x7f6578100ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6568009930 tx=0x7f656802a420 comp rx=0 tx=0).stop 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 -- 172.21.15.3:0/101875814 shutdown_connections 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 --2- 172.21.15.3:0/101875814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f657810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 --2- 172.21.15.3:0/101875814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578100a80 0x7f6578100ee0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 --2- 172.21.15.3:0/101875814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578106a80 0x7f6578106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 -- 172.21.15.3:0/101875814 >> 172.21.15.3:0/101875814 conn(0x7f65780fc7a0 msgr2=0x7f65780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 -- 172.21.15.3:0/101875814 shutdown_connections 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.141+0000 7f657f7b9640 1 -- 172.21.15.3:0/101875814 wait complete. 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 Processor -- start 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 -- start start 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578100a80 0x7f65781984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f65781989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6578110e80 con 0x7f6578106a80 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6578110d00 con 0x7f6578101420 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657f7b9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6578111000 con 0x7f6578100a80 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47272/0 (socket says 172.21.15.3:47272) 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 -- 172.21.15.3:0/1086081169 learned_addr learned my addr 172.21.15.3:0/1086081169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657d555640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578100a80 0x7f65781984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657cd54640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f65781989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 -- 172.21.15.3:0/1086081169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578100a80 msgr2=0x7f65781984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578100a80 0x7f65781984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 -- 172.21.15.3:0/1086081169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 msgr2=0x7f65781989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f65781989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657dd56640 1 -- 172.21.15.3:0/1086081169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657819d450 con 0x7f6578106a80 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.142+0000 7f657cd54640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f65781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f657dd56640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f657000cad0 tx=0x7f657000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f656e7fc640 1 -- 172.21.15.3:0/1086081169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657000ae20 con 0x7f6578106a80 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f656e7fc640 1 -- 172.21.15.3:0/1086081169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6570005020 con 0x7f6578106a80 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f656e7fc640 1 -- 172.21.15.3:0/1086081169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657000f6f0 con 0x7f6578106a80 2024-09-12T02:17:50.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657819d740 con 0x7f6578106a80 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.143+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781a5020 con 0x7f6578106a80 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.145+0000 7f656e7fc640 1 -- 172.21.15.3:0/1086081169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f65700051d0 con 0x7f6578106a80 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 Processor -- start 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 -- start start 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 0x55ca14a98670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14a9afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ca149f4090 con 0x55ca14a8d9d0 2024-09-12T02:17:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ca1499ba70 con 0x55ca14a98bb0 2024-09-12T02:17:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f6581b92740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ca149f0050 con 0x55ca14a8e2f0 2024-09-12T02:17:50.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47274/0 (socket says 172.21.15.3:47274) 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f656e7fc640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 0x7f65640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 -- 172.21.15.3:0/512635193 learned_addr learned my addr 172.21.15.3:0/512635193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657dd56640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 0x7f65640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657d555640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 0x55ca14a98670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14a9afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f656e7fc640 1 -- 172.21.15.3:0/1086081169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f65700cf390 con 0x7f6578106a80 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 -- 172.21.15.3:0/512635193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 msgr2=0x55ca14a98670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 0x55ca14a98670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 -- 172.21.15.3:0/512635193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 msgr2=0x55ca14a9afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14a9afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657cd54640 1 -- 172.21.15.3:0/512635193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca149a8130 con 0x55ca14a8d9d0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.146+0000 7f657d555640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 0x55ca14a98670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.147+0000 7f657cd54640 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6568031550 tx=0x7f6568005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63799a8bfb70004 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.147+0000 7f657dd56640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 0x7f65640a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65700a4160 tx=0x7f6570086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.147+0000 7f65637fe640 1 -- 172.21.15.3:0/512635193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568035050 con 0x55ca14a8d9d0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.147+0000 7f65637fe640 1 -- 172.21.15.3:0/512635193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6568005040 con 0x55ca14a8d9d0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.147+0000 7f65637fe640 1 -- 172.21.15.3:0/512635193 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65680048e0 con 0x55ca14a8d9d0 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- 172.21.15.3:0/512635193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 msgr2=0x55ca14a8ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6568031550 tx=0x7f6568005bb0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- 172.21.15.3:0/512635193 shutdown_connections 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14a9afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8e2f0 0x55ca14a98670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- 172.21.15.3:0/512635193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14a8ddb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- 172.21.15.3:0/512635193 >> 172.21.15.3:0/512635193 conn(0x55ca14a89f60 msgr2=0x55ca14a8a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- 172.21.15.3:0/512635193 shutdown_connections 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- 172.21.15.3:0/512635193 wait complete. 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- start start 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 Processor -- start 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8d9d0 0x55ca14a98470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14a95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ca14a9b780 con 0x55ca14a8e2f0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.148+0000 7f6581b92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ca149aef20 con 0x55ca14a8d9d0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6581b92740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ca14a9b900 con 0x55ca14a98bb0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6581b92740 10 client.0 _ll_register_callbacks cb 0x55ca14978030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47278/0 (socket says 172.21.15.3:47278) 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 -- 172.21.15.3:0/1426073883 learned_addr learned my addr 172.21.15.3:0/1426073883 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657cd54640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8d9d0 0x55ca14a98470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657cd54640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14a95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 -- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 msgr2=0x55ca14a95d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14a95d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 -- 172.21.15.3:0/1426073883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8d9d0 msgr2=0x55ca14a98470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8d9d0 0x55ca14a98470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 -- 172.21.15.3:0/1426073883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca147652b0 con 0x55ca14a8e2f0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f657d555640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f657400be60 tx=0x7f65740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb63de9265c6aef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6560ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574012650 con 0x55ca14a8e2f0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f65740127f0 con 0x55ca14a8e2f0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6568005040 con 0x55ca14a8e2f0 2024-09-12T02:17:50.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65740129f0 con 0x55ca14a8e2f0 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6581b92740 10 client.15033 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.149+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca14aa7180 con 0x55ca14a8e2f0 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6568005040 con 0x55ca14a8e2f0 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6574024050 con 0x55ca14a8e2f0 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 10 client.15033 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:50.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 10 client.15033 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 20 client.15033 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14aa97f0 0x55ca14aaac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 4 client.15033 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca14a01600 con 0x55ca14aa97f0 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.150+0000 7f657cd54640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14aa97f0 0x55ca14aaac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f657cd54640 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14aa97f0 0x55ca14aaac60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73c5958e92a6e5b0 server_cookie=286135536a63d15b in_seq=0 out_seq=0 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6560ff9640 10 client.15033 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6560ff9640 1 -- 172.21.15.3:0/1426073883 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca14a01600 con 0x55ca14aa97f0 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6560ff9640 10 client.15033 handle_command_reply: tid=0 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6581b92740 1 client.15033 shutdown 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 msgr2=0x55ca14a957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.151+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f657400be60 tx=0x7f65740117b0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 shutdown_connections 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14aa97f0 0x55ca14aaac60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14a95d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a957c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 --2- 172.21.15.3:0/1426073883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8d9d0 0x55ca14a98470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 >> 172.21.15.3:0/1426073883 conn(0x55ca14a89f60 msgr2=0x55ca14a8fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6560ff9640 0 client.15033 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:17:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 shutdown_connections 2024-09-12T02:17:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 1 -- 172.21.15.3:0/1426073883 wait complete. 2024-09-12T02:17:50.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.152+0000 7f6581b92740 20 client.15033 trim_cache size 0 max 16384 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 Processor -- start 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 -- start start 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8d9d0 0x55ca14aa9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14ab2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aa8fa0 con 0x55ca14a8e2f0 2024-09-12T02:17:50.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aa9670 con 0x55ca14a98bb0 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f6581b92740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aa9d40 con 0x55ca14a8d9d0 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f657d555640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f657d555640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47290/0 (socket says 172.21.15.3:47290) 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.171+0000 7f657d555640 1 -- 172.21.15.3:0/508887695 learned_addr learned my addr 172.21.15.3:0/508887695 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657cd54640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8d9d0 0x55ca14aa9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657cd54640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14ab2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 -- 172.21.15.3:0/508887695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8d9d0 msgr2=0x55ca14aa9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8d9d0 0x55ca14aa9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 -- 172.21.15.3:0/508887695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 msgr2=0x55ca14ab2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14ab2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 -- 172.21.15.3:0/508887695 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca14a9b890 con 0x55ca14a8e2f0 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657cd54640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14ab2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f657d555640 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f657400c010 tx=0x7f6574011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc82b7e5a1e74d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f6562ffd640 1 -- 172.21.15.3:0/508887695 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574012d90 con 0x55ca14a8e2f0 2024-09-12T02:17:50.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f6562ffd640 1 -- 172.21.15.3:0/508887695 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657402a070 con 0x55ca14a8e2f0 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.172+0000 7f6562ffd640 1 -- 172.21.15.3:0/508887695 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657401bc20 con 0x55ca14a8e2f0 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- 172.21.15.3:0/508887695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 msgr2=0x55ca14a947a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f657400c010 tx=0x7f6574011cb0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- 172.21.15.3:0/508887695 shutdown_connections 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a98bb0 0x55ca14ab2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8e2f0 0x55ca14a947a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 --2- 172.21.15.3:0/508887695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a8d9d0 0x55ca14aa9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- 172.21.15.3:0/508887695 >> 172.21.15.3:0/508887695 conn(0x55ca14a89f60 msgr2=0x55ca14a94090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- 172.21.15.3:0/508887695 shutdown_connections 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- 172.21.15.3:0/508887695 wait complete. 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 -- start start 2024-09-12T02:17:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.173+0000 7f6581b92740 1 Processor -- start 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 0x55ca14aae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14aae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aaaae0 con 0x55ca14a8d9d0 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aaa410 con 0x55ca14a8e2f0 2024-09-12T02:17:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ca14aab1b0 con 0x55ca14a98bb0 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 10 client.0 _ll_register_callbacks cb 0x7f6574005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47302/0 (socket says 172.21.15.3:47302) 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 -- 172.21.15.3:0/4120363161 learned_addr learned my addr 172.21.15.3:0/4120363161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657d555640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 0x55ca14aae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 -- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 msgr2=0x55ca14aae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14aae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 -- 172.21.15.3:0/4120363161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 msgr2=0x55ca14aae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 0x55ca14aae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 -- 172.21.15.3:0/4120363161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca14b11ef0 con 0x55ca14a8d9d0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657d555640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 0x55ca14aae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f657cd54640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6568032ef0 tx=0x7f656802ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea79bf1d1400d808 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6560ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568035040 con 0x55ca14a8d9d0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6568004040 con 0x55ca14a8d9d0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.174+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657402a070 con 0x55ca14a8d9d0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6568004450 con 0x55ca14a8d9d0 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 10 client.15039 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:50.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca14b130b0 con 0x55ca14a8d9d0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657402a070 con 0x55ca14a8d9d0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6568004b40 con 0x55ca14a8d9d0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 10 client.15039 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 20 client.15039 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14b17890 0x55ca14b165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 4 client.15039 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "status"}) -- 0x55ca149fbb10 con 0x55ca14b17890 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.175+0000 7f657d555640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14b17890 0x55ca14b165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f657d555640 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14b17890 0x55ca14b165a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dfb40cb7cc9a033 server_cookie=f8255419d4d76e30 in_seq=0 out_seq=0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6560ff9640 10 client.15039 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6560ff9640 1 -- 172.21.15.3:0/4120363161 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ca149fbb10 con 0x55ca14b17890 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6560ff9640 10 client.15039 handle_command_reply: tid=0 2024-09-12T02:17:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 client.15039 shutdown 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 msgr2=0x55ca14aadbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6568032ef0 tx=0x7f656802ab80 comp rx=0 tx=0).stop 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 shutdown_connections 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55ca14b17890 0x55ca14b165a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca14a98bb0 0x55ca14aae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca14a8e2f0 0x55ca14aae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6560ff9640 0 client.15039 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 --2- 172.21.15.3:0/4120363161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ca14a8d9d0 0x55ca14aadbe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 >> 172.21.15.3:0/4120363161 conn(0x55ca14a89f60 msgr2=0x55ca14a89000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 shutdown_connections 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 1 -- 172.21.15.3:0/4120363161 wait complete. 2024-09-12T02:17:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.176+0000 7f6581b92740 20 client.15039 trim_cache size 0 max 16384 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 73.034690643000005, 2024-09-12T02:17:50.193 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:17:50.194 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:17:50.194 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:17:50.194 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 105.262123543 2024-09-12T02:17:50.194 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:17:50.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 msgr2=0x7f65640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 0x7f65640a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65700a4160 tx=0x7f6570086040 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 msgr2=0x7f657819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f657000cad0 tx=0x7f657000cf90 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 shutdown_connections 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f65640a2c10 0x7f65640a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6578106a80 0x7f657819cd70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6578101420 0x7f65781989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 --2- 172.21.15.3:0/1086081169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6578100a80 0x7f65781984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 >> 172.21.15.3:0/1086081169 conn(0x7f65780fc7a0 msgr2=0x7f65781048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 shutdown_connections 2024-09-12T02:17:50.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:50.177+0000 7f657f7b9640 1 -- 172.21.15.3:0/1086081169 wait complete. 2024-09-12T02:17:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:50 smithi026 ceph-mon[35496]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:50.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:50 smithi003 ceph-mon[31476]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:50.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:50 smithi092 ceph-mon[36089]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:50.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:17:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:51.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 -- 172.21.15.3:0/1970052464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108290 msgr2=0x7f21ec10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/1970052464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108290 0x7f21ec10ec30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009cd0 tx=0x7f21e402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 -- 172.21.15.3:0/1970052464 shutdown_connections 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/1970052464 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec108290 0x7f21ec10ec30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/1970052464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec103740 0x7f21ec107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/1970052464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec102d90 0x7f21ec103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 -- 172.21.15.3:0/1970052464 >> 172.21.15.3:0/1970052464 conn(0x7f21ec0fc7d0 msgr2=0x7f21ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 -- 172.21.15.3:0/1970052464 shutdown_connections 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 -- 172.21.15.3:0/1970052464 wait complete. 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.683+0000 7f21f1c0e640 1 Processor -- start 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 -- start start 2024-09-12T02:17:51.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 0x7f21ec100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec108290 0x7f21ec101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec112150 con 0x7f21ec102d90 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec111fd0 con 0x7f21ec103740 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21f1c0e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1122d0 con 0x7f21ec108290 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47314/0 (socket says 172.21.15.3:47314) 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 -- 172.21.15.3:0/479671801 learned_addr learned my addr 172.21.15.3:0/479671801 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eaffd640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 0x7f21ec100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21ebfff640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec108290 0x7f21ec101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 -- 172.21.15.3:0/479671801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec108290 msgr2=0x7f21ec101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec108290 0x7f21ec101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 -- 172.21.15.3:0/479671801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 msgr2=0x7f21ec100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 0x7f21ec100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eb7fe640 1 -- 172.21.15.3:0/479671801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec1a6b70 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.684+0000 7f21eaffd640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 0x7f21ec100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b4a0 tx=0x7f21dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21e8ff9640 1 -- 172.21.15.3:0/479671801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc014070 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21e8ff9640 1 -- 172.21.15.3:0/479671801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc010af0 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21e8ff9640 1 -- 172.21.15.3:0/479671801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc017d10 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec1a6e60 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.685+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1a7310 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.687+0000 7f21e8ff9640 1 -- 172.21.15.3:0/479671801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f21dc017070 con 0x7f21ec102d90 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.687+0000 7f21f3fe7740 1 Processor -- start 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.687+0000 7f21f3fe7740 1 -- start start 2024-09-12T02:17:51.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15759d0 0x55a2a1575db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a1582fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2a14dc090 con 0x55a2a15762f0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1483a70 con 0x55a2a1580bb0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2a14d8050 con 0x55a2a15759d0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47320/0 (socket says 172.21.15.3:47320) 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 -- 172.21.15.3:0/1043270212 learned_addr learned my addr 172.21.15.3:0/1043270212 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21ebfff640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15759d0 0x55a2a1575db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eaffd640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a1582fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21e8ff9640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 0x7f21d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 -- 172.21.15.3:0/1043270212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15759d0 msgr2=0x55a2a1575db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15759d0 0x55a2a1575db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 -- 172.21.15.3:0/1043270212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 msgr2=0x55a2a1582fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a1582fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21ebfff640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 0x7f21d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 -- 172.21.15.3:0/1043270212 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a1490130 con 0x55a2a15762f0 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eaffd640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a1582fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21e8ff9640 1 -- 172.21.15.3:0/479671801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f21dc0ce4e0 con 0x7f21ec102d90 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.688+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cbf00 tx=0x7f21dc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0c8dbbbe2889be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21d1ffb640 1 -- 172.21.15.3:0/1043270212 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc01e020 con 0x55a2a15762f0 2024-09-12T02:17:51.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21d1ffb640 1 -- 172.21.15.3:0/1043270212 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc09b3e0 con 0x55a2a15762f0 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21d1ffb640 1 -- 172.21.15.3:0/1043270212 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc09f090 con 0x55a2a15762f0 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1043270212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 msgr2=0x55a2a1580670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21ebfff640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 0x7f21d40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009430 tx=0x7f21e402a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cbf00 tx=0x7f21dc0047d0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1043270212 shutdown_connections 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a1582fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15762f0 0x55a2a1580670 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1043270212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15759d0 0x55a2a1575db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1043270212 >> 172.21.15.3:0/1043270212 conn(0x55a2a1571f60 msgr2=0x55a2a1572340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1043270212 shutdown_connections 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.689+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1043270212 wait complete. 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 -- start start 2024-09-12T02:17:51.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 Processor -- start 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15762f0 0x55a2a157d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a1580bb0 0x55a2a157dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1583720 con 0x55a2a15759d0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1496f20 con 0x55a2a15762f0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2a15838a0 con 0x55a2a1580bb0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21f3fe7740 10 client.0 _ll_register_callbacks cb 0x55a2a1460030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47324/0 (socket says 172.21.15.3:47324) 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.690+0000 7f21eb7fe640 1 -- 172.21.15.3:0/779712824 learned_addr learned my addr 172.21.15.3:0/779712824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21ebfff640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a1580bb0 0x55a2a157dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eaffd640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15762f0 0x55a2a157d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 -- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a1580bb0 msgr2=0x55a2a157dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a1580bb0 0x55a2a157dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 -- 172.21.15.3:0/779712824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15762f0 msgr2=0x55a2a157d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15762f0 0x55a2a157d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 -- 172.21.15.3:0/779712824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a124d2b0 con 0x55a2a15759d0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cc010 tx=0x7f21dc0cc300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a87e873373daf59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc01e050 con 0x55a2a15759d0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f21dc010ca0 con 0x55a2a15759d0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc09b3e0 con 0x55a2a15759d0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21dc09a820 con 0x55a2a15759d0 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21f3fe7740 10 client.15051 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a2a158f030 con 0x55a2a15759d0 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc09b3e0 con 0x55a2a15759d0 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.691+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21dc0b2070 con 0x55a2a15759d0 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 10 client.15051 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 20 client.15051 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1591220 0x55a2a1592690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 4 client.15051 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2a14e9600 con 0x55a2a1591220 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21eaffd640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1591220 0x55a2a1592690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21caffd640 10 client.15051 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.692+0000 7f21eaffd640 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1591220 0x55a2a1592690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa260710a4b79b3 server_cookie=767a076e31157337 in_seq=0 out_seq=0 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21caffd640 1 -- 172.21.15.3:0/779712824 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2a14e9600 con 0x55a2a1591220 2024-09-12T02:17:51.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21caffd640 10 client.15051 handle_command_reply: tid=0 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 client.15051 shutdown 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 msgr2=0x55a2a15803d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0cc010 tx=0x7f21dc0cc300 comp rx=0 tx=0).stop 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 shutdown_connections 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1591220 0x55a2a1592690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a1580bb0 0x55a2a157dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15762f0 0x55a2a157d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/779712824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a15803d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21caffd640 0 client.15051 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 >> 172.21.15.3:0/779712824 conn(0x55a2a1571f60 msgr2=0x55a2a1577c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 shutdown_connections 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.693+0000 7f21f3fe7740 1 -- 172.21.15.3:0/779712824 wait complete. 2024-09-12T02:17:51.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.694+0000 7f21f3fe7740 20 client.15051 trim_cache size 0 max 16384 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 Processor -- start 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 -- start start 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1570ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a157a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2a15909d0 con 0x55a2a15759d0 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2a15910a0 con 0x55a2a1580bb0 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1591770 con 0x55a2a15762f0 2024-09-12T02:17:51.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47336/0 (socket says 172.21.15.3:47336) 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eb7fe640 1 -- 172.21.15.3:0/2858952005 learned_addr learned my addr 172.21.15.3:0/2858952005 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eaffd640 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1570ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eb7fe640 1 -- 172.21.15.3:0/2858952005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 msgr2=0x55a2a1570ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.712+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1570ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21eb7fe640 1 -- 172.21.15.3:0/2858952005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 msgr2=0x55a2a157a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21ebfff640 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a157a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a157a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21eb7fe640 1 -- 172.21.15.3:0/2858952005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2a1571d90 con 0x55a2a15759d0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009420 tx=0x7f21dc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4195947900381fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21d17fa640 1 -- 172.21.15.3:0/2858952005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc01e070 con 0x55a2a15759d0 2024-09-12T02:17:51.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21d17fa640 1 -- 172.21.15.3:0/2858952005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc047cd0 con 0x55a2a15759d0 2024-09-12T02:17:51.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.713+0000 7f21d17fa640 1 -- 172.21.15.3:0/2858952005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc047b30 con 0x55a2a15759d0 2024-09-12T02:17:51.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- 172.21.15.3:0/2858952005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 msgr2=0x55a2a1592690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009420 tx=0x7f21dc0047d0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- 172.21.15.3:0/2858952005 shutdown_connections 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a1580bb0 0x55a2a157a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1570ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/2858952005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a15759d0 0x55a2a1592690 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- 172.21.15.3:0/2858952005 >> 172.21.15.3:0/2858952005 conn(0x55a2a1571f60 msgr2=0x55a2a15707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- 172.21.15.3:0/2858952005 shutdown_connections 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- 172.21.15.3:0/2858952005 wait complete. 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 -- start start 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.714+0000 7f21f3fe7740 1 Processor -- start 2024-09-12T02:17:51.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 0x55a2a159afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1596310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1592510 con 0x55a2a1580bb0 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1591e40 con 0x55a2a15759d0 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2a1592be0 con 0x55a2a15762f0 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21f3fe7740 10 client.0 _ll_register_callbacks cb 0x7f21e4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47348/0 (socket says 172.21.15.3:47348) 2024-09-12T02:17:51.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 -- 172.21.15.3:0/1837687574 learned_addr learned my addr 172.21.15.3:0/1837687574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 0x55a2a159afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21eaffd640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1596310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 -- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 msgr2=0x55a2a1596310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1596310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 -- 172.21.15.3:0/1837687574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 msgr2=0x55a2a159afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 0x55a2a159afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 -- 172.21.15.3:0/1837687574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2a1597030 con 0x55a2a1580bb0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 0x55a2a159afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21ebfff640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009cd0 tx=0x7f21e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de23b7be7f3c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.715+0000 7f21caffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40305a0 con 0x55a2a1580bb0 2024-09-12T02:17:51.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f21e4030740 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21dc047cd0 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f21e4030940 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 10 client.15057 fetch_fsmap learned FSMap version 28 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a2a15fb880 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21dc047cd0 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f21e4052070 con 0x55a2a1580bb0 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 10 client.15057 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 20 client.15057 populate_metadata read hostname 'smithi003' 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1600460 0x55a2a15ff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 4 client.15057 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:17:51.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.716+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "status"}) -- 0x55a2a14e3b10 con 0x55a2a1600460 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1600460 0x55a2a15ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21eb7fe640 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1600460 0x55a2a15ff170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0e5dc162238800 server_cookie=35bcc2d0a041bc30 in_seq=0 out_seq=0 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21caffd640 10 client.15057 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21caffd640 1 -- 172.21.15.3:0/1837687574 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a2a14e3b10 con 0x55a2a1600460 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21caffd640 10 client.15057 handle_command_reply: tid=0 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21f3fe7740 1 client.15057 shutdown 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 msgr2=0x55a2a1596850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.717+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009cd0 tx=0x7f21e40095c0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 shutdown_connections 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a2a1600460 0x55a2a15ff170 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2a1580bb0 0x55a2a1596850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2a15762f0 0x55a2a1596310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21caffd640 0 client.15057 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 --2- 172.21.15.3:0/1837687574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2a15759d0 0x55a2a159afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 >> 172.21.15.3:0/1837687574 conn(0x55a2a1571f60 msgr2=0x55a2a1571000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 shutdown_connections 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 1 -- 172.21.15.3:0/1837687574 wait complete. 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f3fe7740 20 client.15057 trim_cache size 0 max 16384 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:17:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24508, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 81.602314864999997, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 101.190333802 2024-09-12T02:17:51.726 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 msgr2=0x7f21d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 0x7f21d40a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009430 tx=0x7f21e402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 msgr2=0x7f21ec1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b4a0 tx=0x7f21dc00b960 comp rx=0 tx=0).stop 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 shutdown_connections 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f21d40a2c10 0x7f21d40a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21ec108290 0x7f21ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ec103740 0x7f21ec100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 --2- 172.21.15.3:0/479671801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f21ec102d90 0x7f21ec1009f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:51.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.718+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 >> 172.21.15.3:0/479671801 conn(0x7f21ec0fc7d0 msgr2=0x7f21ec105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:51.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.719+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 shutdown_connections 2024-09-12T02:17:51.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:51.719+0000 7f21f1c0e640 1 -- 172.21.15.3:0/479671801 wait complete. 2024-09-12T02:17:52.399 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:52 smithi003 ceph-mon[31476]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:52.429 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T02:17:52.429 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:17:52.443 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:17:52.443 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T02:17:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:52 smithi092 ceph-mon[36089]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:52.698 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:52.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:52 smithi026 ceph-mon[35496]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 -- 172.21.15.3:0/2670052062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 msgr2=0x7f700810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 --2- 172.21.15.3:0/2670052062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f700810a050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f700002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 -- 172.21.15.3:0/2670052062 shutdown_connections 2024-09-12T02:17:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 --2- 172.21.15.3:0/2670052062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f700810a050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 --2- 172.21.15.3:0/2670052062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 0x7f7008103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 --2- 172.21.15.3:0/2670052062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 0x7f7008102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 -- 172.21.15.3:0/2670052062 >> 172.21.15.3:0/2670052062 conn(0x7f70080fd390 msgr2=0x7f70080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 -- 172.21.15.3:0/2670052062 shutdown_connections 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 -- 172.21.15.3:0/2670052062 wait complete. 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.311+0000 7f700fb3c640 1 Processor -- start 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 -- start start 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 0x7f700819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 0x7f700819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f700810d880 con 0x7f70081037c0 2024-09-12T02:17:53.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f700810d700 con 0x7f7008102e00 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700fb3c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f700810da00 con 0x7f7008101c00 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47354/0 (socket says 172.21.15.3:47354) 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 -- 172.21.15.3:0/110361133 learned_addr learned my addr 172.21.15.3:0/110361133 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700d0d7640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 0x7f700819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700d8d8640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 0x7f700819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 -- 172.21.15.3:0/110361133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 msgr2=0x7f700819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 0x7f700819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 -- 172.21.15.3:0/110361133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 msgr2=0x7f700819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 0x7f700819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 -- 172.21.15.3:0/110361133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70081a1d00 con 0x7f70081037c0 2024-09-12T02:17:53.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.312+0000 7f700e0d9640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70000098d0 tx=0x7f700002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.313+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700002fc30 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.313+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000003e50 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.313+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000305c0 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.313+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70081a1f90 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.313+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a23f0 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.314+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70081a2710 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.315+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7000004b40 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.316+0000 7f6ffeffd640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 0x7f6ff40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.316+0000 7f700d8d8640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 0x7f6ff40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.316+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f70000f05d0 con 0x7f70081037c0 2024-09-12T02:17:53.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.317+0000 7f700d8d8640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 0x7f6ff40a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70040058c0 tx=0x7f700400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:53.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.320+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70000bd560 con 0x7f70081037c0 2024-09-12T02:17:53.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.511+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f70081a28e0 con 0x7f6ff40a2c40 2024-09-12T02:17:53.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.512+0000 7f6ffeffd640 1 -- 172.21.15.3:0/110361133 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f70081a28e0 con 0x7f6ff40a2c40 2024-09-12T02:17:53.513 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-09-12T02:17:53.513 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-09-12T02:17:53.513 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 msgr2=0x7f6ff40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 0x7f6ff40a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f70040058c0 tx=0x7f700400a4c0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 msgr2=0x7f70081a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70000098d0 tx=0x7f700002f840 comp rx=0 tx=0).stop 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 shutdown_connections 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6ff40a2c40 0x7f6ff40a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70081037c0 0x7f70081a1690 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7008102e00 0x7f700819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 --2- 172.21.15.3:0/110361133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7008101c00 0x7f700819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 >> 172.21.15.3:0/110361133 conn(0x7f70080fd390 msgr2=0x7f7008074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 shutdown_connections 2024-09-12T02:17:53.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:53.514+0000 7f700fb3c640 1 -- 172.21.15.3:0/110361133 wait complete. 2024-09-12T02:17:54.155 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T02:17:54.414 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:54.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:54 smithi003 ceph-mon[31476]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:54.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:54 smithi092 ceph-mon[36089]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:54 smithi026 ceph-mon[35496]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 -- 172.21.15.3:0/1728821262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4102690 msgr2=0x7ff2c4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 --2- 172.21.15.3:0/1728821262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4102690 0x7ff2c4108f20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009ca0 tx=0x7ff2bc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 -- 172.21.15.3:0/1728821262 shutdown_connections 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 --2- 172.21.15.3:0/1728821262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4102690 0x7ff2c4108f20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 --2- 172.21.15.3:0/1728821262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4101cd0 0x7ff2c4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 --2- 172.21.15.3:0/1728821262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4100ad0 0x7ff2c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 -- 172.21.15.3:0/1728821262 >> 172.21.15.3:0/1728821262 conn(0x7ff2c40fc280 msgr2=0x7ff2c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:54.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 -- 172.21.15.3:0/1728821262 shutdown_connections 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.925+0000 7ff2cba96640 1 -- 172.21.15.3:0/1728821262 wait complete. 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 Processor -- start 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 -- start start 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4101cd0 0x7ff2c419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 0x7ff2c419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c410c7b0 con 0x7ff2c4100ad0 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c410c630 con 0x7ff2c4101cd0 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2cba96640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c410c930 con 0x7ff2c4102690 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47360/0 (socket says 172.21.15.3:47360) 2024-09-12T02:17:54.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 -- 172.21.15.3:0/241934313 learned_addr learned my addr 172.21.15.3:0/241934313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2ca033640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 0x7ff2c419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 -- 172.21.15.3:0/241934313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 msgr2=0x7ff2c419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 0x7ff2c419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 -- 172.21.15.3:0/241934313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4101cd0 msgr2=0x7ff2c419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9031640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4101cd0 0x7ff2c419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4101cd0 0x7ff2c419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2c9832640 1 -- 172.21.15.3:0/241934313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c419fb30 con 0x7ff2c4100ad0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.926+0000 7ff2ca033640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 0x7ff2c419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2c9832640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000d770 tx=0x7ff2c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0017070 con 0x7ff2c4100ad0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c0009c30 con 0x7ff2c4100ad0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0003dd0 con 0x7ff2c4100ad0 2024-09-12T02:17:54.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c419fe20 con 0x7ff2c4100ad0 2024-09-12T02:17:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.927+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c41a0300 con 0x7ff2c4100ad0 2024-09-12T02:17:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.928+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c4107ef0 con 0x7ff2c4100ad0 2024-09-12T02:17:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.929+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff2c0003fb0 con 0x7ff2c4100ad0 2024-09-12T02:17:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.930+0000 7ff2baffd640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 0x7ff2b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.930+0000 7ff2c9031640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 0x7ff2b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.930+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff2c00ceff0 con 0x7ff2c4100ad0 2024-09-12T02:17:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.930+0000 7ff2c9031640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 0x7ff2b00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2c406a890 tx=0x7ff2b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:54.933+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2c0009de0 con 0x7ff2c4100ad0 2024-09-12T02:17:55.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.117+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff2c4062630 con 0x7ff2b00a2c90 2024-09-12T02:17:55.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.129+0000 7ff2baffd640 1 -- 172.21.15.3:0/241934313 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7ff2c4062630 con 0x7ff2b00a2c90 2024-09-12T02:17:55.130 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T02:17:55.130 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (99s) 19s ago 98s 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 665495a7f7fa 2024-09-12T02:17:55.130 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi026 running (96s) 28s ago 95s 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 f7ebb6aae9dc 2024-09-12T02:17:55.130 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi092 running (110s) 29s ago 110s 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 3eee5212662c 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (118s) 19s ago 118s 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 8dc3a71df0c6 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi026 running (115s) 28s ago 115s 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 db4d59d415a5 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi092 running (113s) 29s ago 112s 17.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 fa5e311a6b59 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (102s) 19s ago 101s 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 0ab59059144a 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi026 running (107s) 28s ago 107s 17.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 59781e152b2e 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi092 running (104s) 29s ago 104s 25.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 ed0c300e1045 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (2m) 19s ago 2m 13.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 0b6d6896df3e 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi026 running (2m) 28s ago 2m 16.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 61457315352c 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi092 running (2m) 29s ago 2m 17.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 497530ec060b 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (8m) 19s ago 8m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 91cdfa5dea8c 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi026 *:8443,8765 running (7m) 28s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 5ed8fdbe058e 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi092 *:8443,8765 running (7m) 29s ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 9f5bec734895 2024-09-12T02:17:55.131 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (9m) 19s ago 9m 49.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 4fa36850ca4e 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi026 running (7m) 28s ago 7m 38.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 0761ad51343b 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi092 running (7m) 29s ago 7m 42.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 99eca5e2f7c8 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (7m) 19s ago 7m 44.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 5b50a2470d97 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi026 running (6m) 28s ago 6m 55.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 ef79b7e25ce1 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi092 running (6m) 29s ago 6m 56.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1f1e6ec49af4 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (5m) 19s ago 5m 54.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 92c14d2f6ea9 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi026 running (5m) 28s ago 5m 42.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 c61d70f1669e 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi092 running (4m) 29s ago 4m 55.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 dbd339c33e87 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (4m) 19s ago 4m 41.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 2b7b5e3673a9 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi026 running (4m) 28s ago 4m 44.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f001592fc62a 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi092 running (3m) 29s ago 3m 44.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1c69ee7dd72b 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (3m) 19s ago 3m 56.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 12d4436d5b01 2024-09-12T02:17:55.132 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi026 running (2m) 28s ago 2m 44.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 3a7492d4f1c5 2024-09-12T02:17:55.133 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi092 running (2m) 29s ago 2m 40.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 9cbf35ed1f2e 2024-09-12T02:17:55.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 msgr2=0x7ff2b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 0x7ff2b00a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2c406a890 tx=0x7ff2b4008040 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 msgr2=0x7ff2c406ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000d770 tx=0x7ff2c000dc30 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 shutdown_connections 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff2b00a2c90 0x7ff2b00a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff2c4102690 0x7ff2c419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2c4101cd0 0x7ff2c419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 --2- 172.21.15.3:0/241934313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff2c4100ad0 0x7ff2c406ab10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 >> 172.21.15.3:0/241934313 conn(0x7ff2c40fc280 msgr2=0x7ff2c41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 shutdown_connections 2024-09-12T02:17:55.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:55.131+0000 7ff2cba96640 1 -- 172.21.15.3:0/241934313 wait complete. 2024-09-12T02:17:55.441 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:55 smithi003 ceph-mon[31476]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:55.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:55 smithi092 ceph-mon[36089]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:55 smithi026 ceph-mon[35496]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:55.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T02:17:56.064 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:56.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:56 smithi003 ceph-mon[31476]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:56.454 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:56 smithi003 ceph-mon[31476]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:56.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:56 smithi092 ceph-mon[36089]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:56.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:56 smithi092 ceph-mon[36089]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.536+0000 7fe438d5e640 1 -- 172.21.15.3:0/429290448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 msgr2=0x7fe43410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.536+0000 7fe438d5e640 1 --2- 172.21.15.3:0/429290448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe43410a050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009ca0 tx=0x7fe42c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 -- 172.21.15.3:0/429290448 shutdown_connections 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 --2- 172.21.15.3:0/429290448 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe43410a050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 --2- 172.21.15.3:0/429290448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434102e00 0x7fe434103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 --2- 172.21.15.3:0/429290448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434101c00 0x7fe434102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 -- 172.21.15.3:0/429290448 >> 172.21.15.3:0/429290448 conn(0x7fe4340fd390 msgr2=0x7fe4340ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:56.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 -- 172.21.15.3:0/429290448 shutdown_connections 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 -- 172.21.15.3:0/429290448 wait complete. 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 Processor -- start 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 -- start start 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.537+0000 7fe438d5e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434101c00 0x7fe43419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434102e00 0x7fe43419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d860 con 0x7fe4341037c0 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d6e0 con 0x7fe434102e00 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d9e0 con 0x7fe434101c00 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47374/0 (socket says 172.21.15.3:47374) 2024-09-12T02:17:56.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 -- 172.21.15.3:0/1428153014 learned_addr learned my addr 172.21.15.3:0/1428153014 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe43259c640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434101c00 0x7fe43419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe431d9b640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434102e00 0x7fe43419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 -- 172.21.15.3:0/1428153014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434101c00 msgr2=0x7fe43419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434101c00 0x7fe43419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 -- 172.21.15.3:0/1428153014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434102e00 msgr2=0x7fe43419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434102e00 0x7fe43419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 -- 172.21.15.3:0/1428153014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4341a1ce0 con 0x7fe4341037c0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe432d9d640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe42c031ae0 tx=0x7fe42c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c003c30 con 0x7fe4341037c0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe42c003dd0 con 0x7fe4341037c0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4341a1f70 con 0x7fe4341037c0 2024-09-12T02:17:56.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c02fcb0 con 0x7fe4341037c0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.538+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a2470 con 0x7fe4341037c0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.540+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4341a26d0 con 0x7fe4341037c0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.541+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe42c004050 con 0x7fe4341037c0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.542+0000 7fe4237fe640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 0x7fe41c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.542+0000 7fe43259c640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 0x7fe41c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.542+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe42c0f0250 con 0x7fe4341037c0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.542+0000 7fe43259c640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 0x7fe41c0a5180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe434102c60 tx=0x7fe428005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:56.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.546+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe42c0bd270 con 0x7fe4341037c0 2024-09-12T02:17:56.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.729+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe4341a28a0 con 0x7fe41c0a2cc0 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.734+0000 7fe4237fe640 1 -- 172.21.15.3:0/1428153014 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe4341a28a0 con 0x7fe41c0a2cc0 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T02:17:56.735 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 30s ago 2m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi092=c;smithi092=f;smithi092=i;smithi092=l;count:12 2024-09-12T02:17:56.736 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 30s ago 7m smithi003=x;smithi026=y;smithi092=z;count:3 2024-09-12T02:17:56.736 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 30s ago 7m smithi003:172.21.15.3=a;smithi026:172.21.15.26=b;smithi092:172.21.15.92=c;count:3 2024-09-12T02:17:56.736 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 30s ago - 2024-09-12T02:17:56.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 msgr2=0x7fe41c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 0x7fe41c0a5180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe434102c60 tx=0x7fe428005f90 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 msgr2=0x7fe4341a1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe42c031ae0 tx=0x7fe42c031ee0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 shutdown_connections 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe41c0a2cc0 0x7fe41c0a5180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe4341037c0 0x7fe4341a1670 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe434102e00 0x7fe43419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 --2- 172.21.15.3:0/1428153014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe434101c00 0x7fe43419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.736+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 >> 172.21.15.3:0/1428153014 conn(0x7fe4340fd390 msgr2=0x7fe434074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.737+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 shutdown_connections 2024-09-12T02:17:56.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:56.737+0000 7fe438d5e640 1 -- 172.21.15.3:0/1428153014 wait complete. 2024-09-12T02:17:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:56 smithi026 ceph-mon[35496]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:17:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:56 smithi026 ceph-mon[35496]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:57.468 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T02:17:57.729 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 -- 172.21.15.3:0/668519165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 msgr2=0x7f1db4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 --2- 172.21.15.3:0/668519165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db4102150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02a410 comp rx=0 tx=0).stop 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 -- 172.21.15.3:0/668519165 shutdown_connections 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 --2- 172.21.15.3:0/668519165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4102690 0x7f1db4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 --2- 172.21.15.3:0/668519165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db4102150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 --2- 172.21.15.3:0/668519165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4100ad0 0x7f1db4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 -- 172.21.15.3:0/668519165 >> 172.21.15.3:0/668519165 conn(0x7f1db40fc280 msgr2=0x7f1db40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 -- 172.21.15.3:0/668519165 shutdown_connections 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.197+0000 7f1db9088640 1 -- 172.21.15.3:0/668519165 wait complete. 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 Processor -- start 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 -- start start 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 0x7f1db419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4102690 0x7f1db41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410d920 con 0x7f1db4101cd0 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410d7a0 con 0x7f1db4102690 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db9088640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1db410daa0 con 0x7f1db4100ad0 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47396/0 (socket says 172.21.15.3:47396) 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 -- 172.21.15.3:0/4078576102 learned_addr learned my addr 172.21.15.3:0/4078576102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db359e640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4102690 0x7f1db41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:58.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db2d9d640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 0x7f1db419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 -- 172.21.15.3:0/4078576102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 msgr2=0x7f1db419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 0x7f1db419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 -- 172.21.15.3:0/4078576102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4102690 msgr2=0x7f1db41a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4102690 0x7f1db41a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 -- 172.21.15.3:0/4078576102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db41a1df0 con 0x7f1db4101cd0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1db2d9d640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 0x7f1db419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.198+0000 7f1dabfff640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c02aa00 tx=0x7f1d9c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.199+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03f030 con 0x7f1db4101cd0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.199+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c003ce0 con 0x7f1db4101cd0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.199+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db41a2080 con 0x7f1db4101cd0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.199+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035070 con 0x7f1db4101cd0 2024-09-12T02:17:58.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.199+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a2630 con 0x7f1db4101cd0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.200+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db41a2890 con 0x7f1db4101cd0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.201+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1d9c042030 con 0x7f1db4101cd0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.202+0000 7f1db0d99640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 0x7f1da00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.202+0000 7f1db2d9d640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 0x7f1da00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.202+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1d9c0efd40 con 0x7f1db4101cd0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.202+0000 7f1db2d9d640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 0x7f1da00a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1db4101b30 tx=0x7f1da4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:58.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.205+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d9c0bcad0 con 0x7f1db4101cd0 2024-09-12T02:17:58.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.389+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1db410b000 con 0x7f1da00a2cc0 2024-09-12T02:17:58.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:58 smithi003 ceph-mon[31476]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:58.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:58 smithi003 ceph-mon[31476]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.391+0000 7f1db0d99640 1 -- 172.21.15.3:0/4078576102 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f1db410b000 con 0x7f1da00a2cc0 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stdout:smithi026 172.21.15.26 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stdout:smithi092 172.21.15.92 2024-09-12T02:17:58.392 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 msgr2=0x7f1da00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 0x7f1da00a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1db4101b30 tx=0x7f1da4008040 comp rx=0 tx=0).stop 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 msgr2=0x7f1db419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c02aa00 tx=0x7f1d9c032da0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 shutdown_connections 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1da00a2cc0 0x7f1da00a5180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db4102690 0x7f1db41a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1db4101cd0 0x7f1db419a700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 --2- 172.21.15.3:0/4078576102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1db4100ad0 0x7f1db419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:58.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 >> 172.21.15.3:0/4078576102 conn(0x7f1db40fc280 msgr2=0x7f1db41098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:58.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 shutdown_connections 2024-09-12T02:17:58.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:58.393+0000 7f1db9088640 1 -- 172.21.15.3:0/4078576102 wait complete. 2024-09-12T02:17:58.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:58 smithi092 ceph-mon[36089]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:58.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:58 smithi092 ceph-mon[36089]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:58.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:58 smithi026 ceph-mon[35496]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:17:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:58 smithi026 ceph-mon[35496]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:59.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T02:17:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:17:59.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:17:59 smithi026 ceph-mon[35496]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:59.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:17:59 smithi092 ceph-mon[36089]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:59.583 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:17:59 smithi003 ceph-mon[31476]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:17:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 -- 172.21.15.3:0/1200477396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 msgr2=0x7f9f48103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:59.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/1200477396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f48103280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009900 tx=0x7f9f3402a410 comp rx=0 tx=0).stop 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 -- 172.21.15.3:0/1200477396 shutdown_connections 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/1200477396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 0x7f9f4810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/1200477396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f48103280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/1200477396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 0x7f9f48102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 -- 172.21.15.3:0/1200477396 >> 172.21.15.3:0/1200477396 conn(0x7f9f480fd390 msgr2=0x7f9f480ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 -- 172.21.15.3:0/1200477396 shutdown_connections 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 -- 172.21.15.3:0/1200477396 wait complete. 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.798+0000 7f9f4ef82640 1 Processor -- start 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 -- start start 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 0x7f9f48197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 0x7f9f4819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:59.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4810d880 con 0x7f9f48102e00 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4810d700 con 0x7f9f481037c0 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4810da00 con 0x7f9f48101c00 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44002/0 (socket says 172.21.15.3:44002) 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 -- 172.21.15.3:0/2177462486 learned_addr learned my addr 172.21.15.3:0/2177462486 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4cd1e640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 0x7f9f48197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4d51f640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 0x7f9f4819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 -- 172.21.15.3:0/2177462486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 msgr2=0x7f9f48197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 0x7f9f48197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 -- 172.21.15.3:0/2177462486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 msgr2=0x7f9f4819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 0x7f9f4819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 -- 172.21.15.3:0/2177462486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4819fb90 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f47fff640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009230 tx=0x7f9f34004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3403f030 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f34003ce0 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.799+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4819fe20 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.800+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34035070 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.800+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481a0280 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.801+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f48108fd0 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.802+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9f34042030 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.803+0000 7f9f45ffb640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 0x7f9f380a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.803+0000 7f9f4cd1e640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 0x7f9f380a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.803+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9f340ef7d0 con 0x7f9f48102e00 2024-09-12T02:17:59.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.803+0000 7f9f4cd1e640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 0x7f9f380a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005fd0 tx=0x7f9f3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:17:59.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.807+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f340bc560 con 0x7f9f48102e00 2024-09-12T02:17:59.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.990+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9f48062630 con 0x7f9f380a2cc0 2024-09-12T02:17:59.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.992+0000 7f9f45ffb640 1 -- 172.21.15.3:0/2177462486 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9f48062630 con 0x7f9f380a2cc0 2024-09-12T02:17:59.993 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T02:17:59.993 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:17:59.994 INFO:teuthology.orchestra.run.smithi003.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:17:59.994 INFO:teuthology.orchestra.run.smithi003.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:17:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 msgr2=0x7f9f380a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 0x7f9f380a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005fd0 tx=0x7f9f3c00b390 comp rx=0 tx=0).stop 2024-09-12T02:17:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 msgr2=0x7f9f481984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:17:59.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009230 tx=0x7f9f34004360 comp rx=0 tx=0).stop 2024-09-12T02:17:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 shutdown_connections 2024-09-12T02:17:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9f380a2cc0 0x7f9f380a5180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f481037c0 0x7f9f4819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9f48102e00 0x7f9f481984a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 --2- 172.21.15.3:0/2177462486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f48101c00 0x7f9f48197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:17:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 >> 172.21.15.3:0/2177462486 conn(0x7f9f480fd390 msgr2=0x7f9f481061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:17:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.994+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 shutdown_connections 2024-09-12T02:17:59.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:17:59.995+0000 7f9f4ef82640 1 -- 172.21.15.3:0/2177462486 wait complete. 2024-09-12T02:18:00.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:00 smithi092 ceph-mon[36089]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:00.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:00 smithi003 ceph-mon[31476]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:00.694 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:18:00.705 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:18:00.705 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T02:18:00.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:00 smithi026 ceph-mon[35496]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:01.340 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:01 smithi003 ceph-mon[31476]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:01.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 -- 172.21.15.3:0/2522610486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac1037c0 msgr2=0x7f74ac10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 --2- 172.21.15.3:0/2522610486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac1037c0 0x7f74ac10a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f74a4009ca0 tx=0x7f74a402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 -- 172.21.15.3:0/2522610486 shutdown_connections 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 --2- 172.21.15.3:0/2522610486 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac1037c0 0x7f74ac10a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 --2- 172.21.15.3:0/2522610486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac102e00 0x7f74ac103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 --2- 172.21.15.3:0/2522610486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac101c00 0x7f74ac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 -- 172.21.15.3:0/2522610486 >> 172.21.15.3:0/2522610486 conn(0x7f74ac0fd390 msgr2=0x7f74ac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.421+0000 7f74b0844640 1 -- 172.21.15.3:0/2522610486 shutdown_connections 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 -- 172.21.15.3:0/2522610486 wait complete. 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 Processor -- start 2024-09-12T02:18:01.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 -- start start 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac101c00 0x7f74ac075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac1037c0 0x7f74ac074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10d8b0 con 0x7f74ac102e00 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10d730 con 0x7f74ac101c00 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.422+0000 7f74b0844640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac10da30 con 0x7f74ac1037c0 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44022/0 (socket says 172.21.15.3:44022) 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 -- 172.21.15.3:0/1101589163 learned_addr learned my addr 172.21.15.3:0/1101589163 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74aa59c640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac101c00 0x7f74ac075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:01.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74aad9d640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac1037c0 0x7f74ac074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 -- 172.21.15.3:0/1101589163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac1037c0 msgr2=0x7f74ac074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac1037c0 0x7f74ac074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 -- 172.21.15.3:0/1101589163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac101c00 msgr2=0x7f74ac075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac101c00 0x7f74ac075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 -- 172.21.15.3:0/1101589163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac0753e0 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74a9d9b640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74a000d950 tx=0x7f74a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0017070 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a0009c30 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1a1fe0 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.423+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000f6b0 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.424+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a2440 con 0x7f74ac102e00 2024-09-12T02:18:01.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.424+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac108f30 con 0x7f74ac102e00 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.426+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f74a00109d0 con 0x7f74ac102e00 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.426+0000 7f749b7fe640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 0x7f74940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.427+0000 7f74aa59c640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 0x7f74940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.427+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f74a00cedb0 con 0x7f74ac102e00 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.427+0000 7f74aa59c640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 0x7f74940a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:01.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.431+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74a0009de0 con 0x7f74ac102e00 2024-09-12T02:18:01.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.644+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f74ac02d9e0 con 0x7f74ac102e00 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.645+0000 7f749b7fe640 1 -- 172.21.15.3:0/1101589163 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3110 (secure 0 0 0) 0x7f74a009bba0 con 0x7f74ac102e00 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout:e28 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout:btime 2024-09-12T02:16:41:743221+0000 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2024-09-12T02:18:01.646 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 28 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-12T02:16:29.718750+0000 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-09-12T02:16:40.792890+0000 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2024-09-12T02:18:01.647 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 5 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:in 0,1,2,3,4 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=24508,1=24493,2=24487,3=24482,4=14619} 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3] 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2024-09-12T02:18:01.648 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:qdb_cluster leader: 24508 members: 14619,24482,24508,24487,24493 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{0:24508} state up:active seq 6 addr [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.26:6836/2891911183,v1:172.21.15.26:6837/2891911183] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{1:24493} state up:active seq 10 addr [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.92:6832/3877701831,v1:172.21.15.92:6833/3877701831] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{2:24487} state up:active seq 10 addr [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{2:24425} state up:standby-replay seq 1 addr [v2:172.21.15.26:6834/1866420590,v1:172.21.15.26:6835/1866420590] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{3:24482} state up:active seq 6 addr [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{3:24404} state up:standby-replay seq 1 addr [v2:172.21.15.26:6832/1804609961,v1:172.21.15.26:6833/1804609961] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{4:14619} state up:active seq 7 addr [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.649 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{4:14613} state up:standby-replay seq 1 addr [v2:172.21.15.3:6838/775355044,v1:172.21.15.3:6839/775355044] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.3:6834/3477371464,v1:172.21.15.3:6835/3477371464] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.3:6836/4187721968,v1:172.21.15.3:6837/4187721968] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:18:01.650 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 msgr2=0x7f74940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 0x7f74940a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f749c005fd0 tx=0x7f749c00b410 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 msgr2=0x7f74ac0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f74a000d950 tx=0x7f74a000de10 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 shutdown_connections 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f74940a2c10 0x7f74940a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74ac1037c0 0x7f74ac074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f74ac102e00 0x7f74ac0745e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 --2- 172.21.15.3:0/1101589163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74ac101c00 0x7f74ac075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 >> 172.21.15.3:0/1101589163 conn(0x7f74ac0fd390 msgr2=0x7f74ac106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 shutdown_connections 2024-09-12T02:18:01.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:01.647+0000 7f74b0844640 1 -- 172.21.15.3:0/1101589163 wait complete. 2024-09-12T02:18:01.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:01 smithi026 ceph-mon[35496]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:01 smithi092 ceph-mon[36089]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:02.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:02 smithi003 ceph-mon[31476]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:02.367 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:02 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1101589163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:18:02.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T02:18:02.628 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:02 smithi026 ceph-mon[35496]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:02 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1101589163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:18:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:02 smithi092 ceph-mon[36089]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:02 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1101589163' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:18:03.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 -- 172.21.15.3:0/474452832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f241037c0 msgr2=0x7f7f2410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/474452832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f241037c0 0x7f7f2410a050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009ca0 tx=0x7f7f1c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 -- 172.21.15.3:0/474452832 shutdown_connections 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/474452832 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f241037c0 0x7f7f2410a050 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/474452832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24102e00 0x7f7f24103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/474452832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f24101c00 0x7f7f24102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.116+0000 7f7f29ddd640 1 -- 172.21.15.3:0/474452832 >> 172.21.15.3:0/474452832 conn(0x7f7f240fd390 msgr2=0x7f7f240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.117+0000 7f7f29ddd640 1 -- 172.21.15.3:0/474452832 shutdown_connections 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.117+0000 7f7f29ddd640 1 -- 172.21.15.3:0/474452832 wait complete. 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.117+0000 7f7f29ddd640 1 Processor -- start 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 -- start start 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24101c00 0x7f7f24197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:03.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f241037c0 0x7f7f2419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410d860 con 0x7f7f24102e00 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410d6e0 con 0x7f7f24101c00 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f29ddd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2410d9e0 con 0x7f7f241037c0 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44040/0 (socket says 172.21.15.3:44040) 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 -- 172.21.15.3:0/3651728770 learned_addr learned my addr 172.21.15.3:0/3651728770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f23fff640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f241037c0 0x7f7f2419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f237fe640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24101c00 0x7f7f24197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 -- 172.21.15.3:0/3651728770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f241037c0 msgr2=0x7f7f2419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f241037c0 0x7f7f2419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 -- 172.21.15.3:0/3651728770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24101c00 msgr2=0x7f7f24197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24101c00 0x7f7f24197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 -- 172.21.15.3:0/3651728770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419fab0 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.118+0000 7f7f22ffd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.119+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800cd80 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.119+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f18010910 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.119+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2419fda0 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.119+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18018990 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.119+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a0250 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.120+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f24109000 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.121+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7f18010ac0 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.121+0000 7f7f20ff9640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 0x7f7f0c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.122+0000 7f7f237fe640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 0x7f7f0c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.122+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7f180cf480 con 0x7f7f24102e00 2024-09-12T02:18:03.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.122+0000 7f7f237fe640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 0x7f7f0c0a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f24102c60 tx=0x7f7f14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:03.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.126+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f1809bc20 con 0x7f7f24102e00 2024-09-12T02:18:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.304+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7f2402da10 con 0x7f7f24102e00 2024-09-12T02:18:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.305+0000 7f7f20ff9640 1 -- 172.21.15.3:0/3651728770 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5242 (secure 0 0 0) 0x7f7f1809ff20 con 0x7f7f24102e00 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 83 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:fsid c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-09-12T02:08:39.465171+0000 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-09-12T02:16:38.132385+0000 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 26 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2024-09-12T02:18:03.306 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release squid 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/46072928,v1:172.21.15.3:6803/46072928] [v2:172.21.15.3:6804/46072928,v1:172.21.15.3:6805/46072928] exists,up 9d0aab17-f2a0-467b-9ab2-ddd8fc92d297 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6800/2606408940,v1:172.21.15.26:6801/2606408940] [v2:172.21.15.26:6802/2606408940,v1:172.21.15.26:6803/2606408940] exists,up dcaeda7b-e940-4d6e-84b7-4b839bed3275 2024-09-12T02:18:03.307 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 17 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/3785909274,v1:172.21.15.92:6801/3785909274] [v2:172.21.15.92:6802/3785909274,v1:172.21.15.92:6803/3785909274] exists,up 6e76989d-dcc2-45a3-8f36-eb08674b7fdc 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 24 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/278455203,v1:172.21.15.3:6811/278455203] [v2:172.21.15.3:6812/278455203,v1:172.21.15.3:6813/278455203] exists,up f79efe58-167d-445a-a4c8-47c4f46c1070 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6808/525329948,v1:172.21.15.26:6809/525329948] [v2:172.21.15.26:6810/525329948,v1:172.21.15.26:6811/525329948] exists,up f3e2dea9-afc3-4c05-9c71-75758b7b311b 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2340798273,v1:172.21.15.92:6809/2340798273] [v2:172.21.15.92:6810/2340798273,v1:172.21.15.92:6811/2340798273] exists,up 6a9b28bd-0597-4710-b614-d90867985f7f 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/342811658,v1:172.21.15.3:6819/342811658] [v2:172.21.15.3:6820/342811658,v1:172.21.15.3:6821/342811658] exists,up b1c4272b-d682-4423-924c-8500633b6e24 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6816/500611310,v1:172.21.15.26:6817/500611310] [v2:172.21.15.26:6818/500611310,v1:172.21.15.26:6819/500611310] exists,up 3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/1598499134,v1:172.21.15.92:6817/1598499134] [v2:172.21.15.92:6818/1598499134,v1:172.21.15.92:6819/1598499134] exists,up e2c1241a-ec0f-4db8-b5d4-0e8895ddc469 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/3263587345,v1:172.21.15.3:6827/3263587345] [v2:172.21.15.3:6828/3263587345,v1:172.21.15.3:6829/3263587345] exists,up 7611642c-8229-45c0-b752-967b7260e2ae 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6824/3810836866,v1:172.21.15.26:6825/3810836866] [v2:172.21.15.26:6826/3810836866,v1:172.21.15.26:6827/3810836866] exists,up 1ab8f17b-02b0-441a-bddf-77679403a160 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 75 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/3259822775,v1:172.21.15.92:6825/3259822775] [v2:172.21.15.92:6826/3259822775,v1:172.21.15.92:6827/3259822775] exists,up 84dbc141-f1cc-4f68-909e-53f67c8e5190 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.4 [2,11] 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.6 [8,11] 2024-09-12T02:18:03.308 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.6 [3,7] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.b [3,9] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.11 [4,8,5,9] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.13 [4,9] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.21 [4,6] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.23 [4,8] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.39 [5,6] 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2921615271 expires 2024-09-13T02:09:36.346826+0000 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2551485583 expires 2024-09-13T02:09:36.346826+0000 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/1539065682 expires 2024-09-13T02:09:36.346826+0000 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2796707951 expires 2024-09-13T02:09:12.115573+0000 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1673598194 expires 2024-09-13T02:09:36.346826+0000 2024-09-12T02:18:03.309 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/814335086 expires 2024-09-13T02:09:12.115573+0000 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/1539065682 expires 2024-09-13T02:09:36.346826+0000 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/814335086 expires 2024-09-13T02:09:12.115573+0000 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2395673224 expires 2024-09-13T02:09:12.115573+0000 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3787188077 expires 2024-09-13T02:09:12.115573+0000 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.306+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 msgr2=0x7f7f0c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:03.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 0x7f7f0c0a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f24102c60 tx=0x7f7f14005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 msgr2=0x7f7f241983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800e9f0 tx=0x7f7f1800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 shutdown_connections 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7f0c0a2c40 0x7f7f0c0a5100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f241037c0 0x7f7f2419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7f24102e00 0x7f7f241983c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 --2- 172.21.15.3:0/3651728770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f24101c00 0x7f7f24197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 >> 172.21.15.3:0/3651728770 conn(0x7f7f240fd390 msgr2=0x7f7f241061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 shutdown_connections 2024-09-12T02:18:03.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:03.307+0000 7f7f29ddd640 1 -- 172.21.15.3:0/3651728770 wait complete. 2024-09-12T02:18:03.988 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T02:18:04.000 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:04.260 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:04.675 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:04 smithi003 ceph-mon[31476]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:04.676 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:04 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3651728770' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:18:04.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:04 smithi026 ceph-mon[35496]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:04 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3651728770' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:18:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 -- 172.21.15.3:0/1038214441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa108102d40 msgr2=0x7fa108103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 --2- 172.21.15.3:0/1038214441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa108102d40 0x7fa108103120 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa104009900 tx=0x7fa10402a480 comp rx=0 tx=0).stop 2024-09-12T02:18:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 -- 172.21.15.3:0/1038214441 shutdown_connections 2024-09-12T02:18:04.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 --2- 172.21.15.3:0/1038214441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10810c170 0x7fa10810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 --2- 172.21.15.3:0/1038214441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108103660 0x7fa10810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 --2- 172.21.15.3:0/1038214441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa108102d40 0x7fa108103120 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 -- 172.21.15.3:0/1038214441 >> 172.21.15.3:0/1038214441 conn(0x7fa1080fc7a0 msgr2=0x7fa1080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 -- 172.21.15.3:0/1038214441 shutdown_connections 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.755+0000 7fa110611640 1 -- 172.21.15.3:0/1038214441 wait complete. 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 Processor -- start 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 -- start start 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108102d40 0x7fa108198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa108103660 0x7fa108198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111fb0 con 0x7fa10810c170 2024-09-12T02:18:04.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa108111e30 con 0x7fa108103660 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa110611640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112130 con 0x7fa108102d40 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa10ebae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa10ebae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44062/0 (socket says 172.21.15.3:44062) 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa10ebae640 1 -- 172.21.15.3:0/2955951385 learned_addr learned my addr 172.21.15.3:0/2955951385 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa10dbac640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa108103660 0x7fa108198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.756+0000 7fa10e3ad640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108102d40 0x7fa108198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 -- 172.21.15.3:0/2955951385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108102d40 msgr2=0x7fa108198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108102d40 0x7fa108198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 -- 172.21.15.3:0/2955951385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa108103660 msgr2=0x7fa108198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa108103660 0x7fa108198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 -- 172.21.15.3:0/2955951385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819d520 con 0x7fa10810c170 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa10ebae640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa10000cad0 tx=0x7fa10000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:04.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000ae20 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819d810 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa100005020 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000f6f0 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.757+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a50f0 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.758+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108103120 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.760+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa1000051d0 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.760+0000 7fa0ff7fe640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 0x7fa0f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.761+0000 7fa10e3ad640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 0x7fa0f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.761+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa1000cf650 con 0x7fa10810c170 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.761+0000 7fa10e3ad640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 0x7fa0f40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1040096c0 tx=0x7fa104031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:04.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.764+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa100014030 con 0x7fa10810c170 2024-09-12T02:18:04.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:04 smithi092 ceph-mon[36089]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:04.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:04 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3651728770' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:18:04.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.978+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa108107240 con 0x7fa10810c170 2024-09-12T02:18:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.979+0000 7fa0ff7fe640 1 -- 172.21.15.3:0/2955951385 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7fa10009c3e0 con 0x7fa10810c170 2024-09-12T02:18:04.981 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:04.981 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:18:04.981 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:18:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 msgr2=0x7fa0f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 0x7fa0f40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa1040096c0 tx=0x7fa104031490 comp rx=0 tx=0).stop 2024-09-12T02:18:04.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 msgr2=0x7fa10819ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa10000cad0 tx=0x7fa10000cf90 comp rx=0 tx=0).stop 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 shutdown_connections 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa0f40a2c10 0x7fa0f40a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa10810c170 0x7fa10819ce40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa108103660 0x7fa108198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 --2- 172.21.15.3:0/2955951385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa108102d40 0x7fa108198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 >> 172.21.15.3:0/2955951385 conn(0x7fa1080fc7a0 msgr2=0x7fa1080fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 shutdown_connections 2024-09-12T02:18:04.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:04.981+0000 7fa110611640 1 -- 172.21.15.3:0/2955951385 wait complete. 2024-09-12T02:18:05.739 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:05 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2955951385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:05.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:05 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2955951385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:05.770 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:05.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:05 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2955951385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:06.029 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:06.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:06 smithi003 ceph-mon[31476]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 -- 172.21.15.3:0/960202460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 msgr2=0x7f13d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 --2- 172.21.15.3:0/960202460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d0101200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009900 tx=0x7f13c002a480 comp rx=0 tx=0).stop 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 -- 172.21.15.3:0/960202460 shutdown_connections 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 --2- 172.21.15.3:0/960202460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 0x7f13d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 --2- 172.21.15.3:0/960202460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 0x7f13d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 --2- 172.21.15.3:0/960202460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d0101200 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 -- 172.21.15.3:0/960202460 >> 172.21.15.3:0/960202460 conn(0x7f13d00fc7d0 msgr2=0x7f13d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 -- 172.21.15.3:0/960202460 shutdown_connections 2024-09-12T02:18:06.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.540+0000 7f13d4e14640 1 -- 172.21.15.3:0/960202460 wait complete. 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 Processor -- start 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 -- start start 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 0x7f13d0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 0x7f13d01a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0110f50 con 0x7f13d0100e20 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0110dd0 con 0x7f13d0102ef0 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13d4e14640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f13d01110d0 con 0x7f13d0102570 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13ce59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44080/0 (socket says 172.21.15.3:44080) 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13ce59c640 1 -- 172.21.15.3:0/870070573 learned_addr learned my addr 172.21.15.3:0/870070573 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:06.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13ced9d640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 0x7f13d01a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.541+0000 7f13cdd9b640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 0x7f13d0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 -- 172.21.15.3:0/870070573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 msgr2=0x7f13d0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 0x7f13d0199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 -- 172.21.15.3:0/870070573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 msgr2=0x7f13d01a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 0x7f13d01a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 -- 172.21.15.3:0/870070573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d01a4980 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13ce59c640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006010 tx=0x7f13c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c003f070 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c0031c30 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d01a4b50 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0035030 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.542+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01a4fa0 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.543+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d0101200 con 0x7f13d0100e20 2024-09-12T02:18:06.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.544+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f13c0042070 con 0x7f13d0100e20 2024-09-12T02:18:06.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.545+0000 7f13bf7fe640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:06.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.545+0000 7f13cdd9b640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:06.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.545+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f13c00efd80 con 0x7f13d0100e20 2024-09-12T02:18:06.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.546+0000 7f13cdd9b640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 0x7f13b80a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13d006aac0 tx=0x7f13c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:06.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.549+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13c0031de0 con 0x7f13d0100e20 2024-09-12T02:18:06.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:06 smithi026 ceph-mon[35496]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:06.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.779+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13d0109090 con 0x7f13d0100e20 2024-09-12T02:18:06.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.780+0000 7f13bf7fe640 1 -- 172.21.15.3:0/870070573 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11946 (secure 0 0 0) 0x7f13c00bc520 con 0x7f13d0100e20 2024-09-12T02:18:06.781 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:06.781 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":28,"btime":"2024-09-12T02:16:41:743221+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:16:40.792890+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:18:06.782 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 28 2024-09-12T02:18:06.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 msgr2=0x7f13b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 0x7f13b80a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13d006aac0 tx=0x7f13c400b410 comp rx=0 tx=0).stop 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 msgr2=0x7f13d01993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006010 tx=0x7f13c0003c10 comp rx=0 tx=0).stop 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 shutdown_connections 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13d0102ef0 0x7f13d01a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d0102570 0x7f13d0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 --2- 172.21.15.3:0/870070573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13d0100e20 0x7f13d01993c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.782+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 >> 172.21.15.3:0/870070573 conn(0x7f13d00fc7d0 msgr2=0x7f13d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.783+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 shutdown_connections 2024-09-12T02:18:06.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:06.783+0000 7f13d4e14640 1 -- 172.21.15.3:0/870070573 wait complete. 2024-09-12T02:18:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:06 smithi092 ceph-mon[36089]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:07.512 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:18:07.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:07 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/870070573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:07.770 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:07.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:07 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/870070573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:07.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:07 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/870070573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 -- 172.21.15.3:0/1288419659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 msgr2=0x7f173c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 --2- 172.21.15.3:0/1288419659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c10d950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1734009cd0 tx=0x7f173402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 -- 172.21.15.3:0/1288419659 shutdown_connections 2024-09-12T02:18:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 --2- 172.21.15.3:0/1288419659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c10d950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 --2- 172.21.15.3:0/1288419659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c100a80 0x7f173c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 --2- 172.21.15.3:0/1288419659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c106a80 0x7f173c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 -- 172.21.15.3:0/1288419659 >> 172.21.15.3:0/1288419659 conn(0x7f173c0fc7a0 msgr2=0x7f173c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 -- 172.21.15.3:0/1288419659 shutdown_connections 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.235+0000 7f17431bf640 1 -- 172.21.15.3:0/1288419659 wait complete. 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 Processor -- start 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 -- start start 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 0x7f173c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 0x7f173c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f173c110e80 con 0x7f173c101420 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f173c110d00 con 0x7f173c106a80 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f17431bf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f173c111000 con 0x7f173c100a80 2024-09-12T02:18:08.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f173bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f173bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44092/0 (socket says 172.21.15.3:44092) 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f173bfff640 1 -- 172.21.15.3:0/4257676968 learned_addr learned my addr 172.21.15.3:0/4257676968 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.236+0000 7f174175c640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 0x7f173c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f1740f5b640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 0x7f173c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 -- 172.21.15.3:0/4257676968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 msgr2=0x7f173c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 0x7f173c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 -- 172.21.15.3:0/4257676968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 msgr2=0x7f173c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 0x7f173c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 -- 172.21.15.3:0/4257676968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c19c1f0 con 0x7f173c101420 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f174175c640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 0x7f173c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f1740f5b640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 0x7f173c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f173bfff640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c920 tx=0x7f172c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:08.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c007c70 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c007e10 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c19c4e0 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c018890 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.237+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c1a3dc0 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.238+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173c100ee0 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.239+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f172c018a70 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.240+0000 7f1739ffb640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 0x7f17280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.240+0000 7f1740f5b640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 0x7f17280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.240+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f172c0cf280 con 0x7f173c101420 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.240+0000 7f1740f5b640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 0x7f17280a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f172400baa0 tx=0x7f1724005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:08.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.244+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f172c014030 con 0x7f173c101420 2024-09-12T02:18:08.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.422+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f173c1a3ff0 con 0x7f173c101420 2024-09-12T02:18:08.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:08 smithi003 ceph-mon[31476]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:08.423 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:08 smithi003 ceph-mon[31476]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:18:08.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.424+0000 7f1739ffb640 1 -- 172.21.15.3:0/4257676968 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f172c09c170 con 0x7f173c101420 2024-09-12T02:18:08.425 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:08.425 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.426+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 msgr2=0x7f17280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.426+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 0x7f17280a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f172400baa0 tx=0x7f1724005d20 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 msgr2=0x7f173c197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c920 tx=0x7f172c00cde0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 shutdown_connections 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f17280a2c00 0x7f17280a50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f173c106a80 0x7f173c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f173c101420 0x7f173c197780 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 --2- 172.21.15.3:0/4257676968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f173c100a80 0x7f173c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 >> 172.21.15.3:0/4257676968 conn(0x7f173c0fc7a0 msgr2=0x7f173c0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 shutdown_connections 2024-09-12T02:18:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:08.427+0000 7f17431bf640 1 -- 172.21.15.3:0/4257676968 wait complete. 2024-09-12T02:18:08.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:08 smithi026 ceph-mon[35496]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:08.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:08 smithi026 ceph-mon[35496]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:18:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:08 smithi092 ceph-mon[36089]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:18:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:08 smithi092 ceph-mon[36089]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:18:09.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:16:29.718750+0000', 'modified': '2024-09-12T02:16:40.792890+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24508, 'mds_1': 24493, 'mds_2': 24487, 'mds_3': 24482, 'mds_4': 14619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/775355044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 775355044}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 775355044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.3:6841/3157578506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 3157578506}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 3157578506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6833/1804609961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1804609961}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 1804609961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1866420590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1866420590}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1866420590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/3877701831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3877701831}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3877701831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6837/2891911183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 2891911183}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2891911183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6839/1997151172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 1997151172}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 1997151172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6835/1180218932', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1180218932}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1180218932}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6837/487477954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 487477954}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 487477954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/1571545450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1571545450}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1571545450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [14619, 24482, 24508, 24487, 24493]} 2024-09-12T02:18:09.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:18:09.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-12T02:18:09.441 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:09.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:09 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4257676968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:09.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:09 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4257676968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:09.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:09 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4257676968' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 -- 172.21.15.3:0/1192861134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80102570 msgr2=0x7f2e801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 --2- 172.21.15.3:0/1192861134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80102570 0x7f2e801029b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009900 tx=0x7f2e6c02a480 comp rx=0 tx=0).stop 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 -- 172.21.15.3:0/1192861134 shutdown_connections 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 --2- 172.21.15.3:0/1192861134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102ef0 0x7f2e8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 --2- 172.21.15.3:0/1192861134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80102570 0x7f2e801029b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 --2- 172.21.15.3:0/1192861134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80100e20 0x7f2e80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 -- 172.21.15.3:0/1192861134 >> 172.21.15.3:0/1192861134 conn(0x7f2e800fc7d0 msgr2=0x7f2e800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 -- 172.21.15.3:0/1192861134 shutdown_connections 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.868+0000 7f2e86e44640 1 -- 172.21.15.3:0/1192861134 wait complete. 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 Processor -- start 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 -- start start 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 0x7f2e8010fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 0x7f2e8010ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80112060 con 0x7f2e80100e20 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2e80111ee0 con 0x7f2e80102ef0 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.869+0000 7f2e86e44640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e801121e0 con 0x7f2e80102570 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60268/0 (socket says 172.21.15.3:60268) 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 -- 172.21.15.3:0/4201274250 learned_addr learned my addr 172.21.15.3:0/4201274250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e853e1640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 0x7f2e8010ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 0x7f2e8010fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 -- 172.21.15.3:0/4201274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 msgr2=0x7f2e8010fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 0x7f2e8010fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 -- 172.21.15.3:0/4201274250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 msgr2=0x7f2e8010ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 0x7f2e8010ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 -- 172.21.15.3:0/4201274250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e801a6a20 con 0x7f2e80100e20 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e853e1640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 0x7f2e8010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 0x7f2e8010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400d950 tx=0x7f2e7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e7dffb640 1 -- 172.21.15.3:0/4201274250 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e74017030 con 0x7f2e80100e20 2024-09-12T02:18:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e801a6d10 con 0x7f2e80100e20 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e7dffb640 1 -- 172.21.15.3:0/4201274250 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e74004020 con 0x7f2e80100e20 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.870+0000 7f2e7dffb640 1 -- 172.21.15.3:0/4201274250 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e740047a0 con 0x7f2e80100e20 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.871+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e801a7240 con 0x7f2e80100e20 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.873+0000 7f2e7dffb640 1 -- 172.21.15.3:0/4201274250 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2e740041d0 con 0x7f2e80100e20 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.873+0000 7f2e8921d740 1 Processor -- start 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.873+0000 7f2e8921d740 1 -- start start 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.873+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.873+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b65670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b67fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e8921d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e756ac1090 con 0x55e756b5a9d0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e8921d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e756a68a70 con 0x55e756b5b2f0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e8921d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e756abd050 con 0x55e756b65bb0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60280/0 (socket says 172.21.15.3:60280) 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 -- 172.21.15.3:0/3270702824 learned_addr learned my addr 172.21.15.3:0/3270702824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e7dffb640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e84be0640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b65670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b67fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e7dffb640 1 -- 172.21.15.3:0/4201274250 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2e740ceb40 con 0x7f2e80100e20 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 -- 172.21.15.3:0/3270702824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 msgr2=0x55e756b67fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b67fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 -- 172.21.15.3:0/3270702824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 msgr2=0x55e756b65670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b65670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 -- 172.21.15.3:0/3270702824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e756a75130 con 0x55e756b5a9d0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b67fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e78013aa0 tx=0x7f2e78013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395faa0f09ad937c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.874+0000 7f2e853e1640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 0x7f2e700a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e78013fd0 tx=0x7f2e7801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e6affd640 1 -- 172.21.15.3:0/3270702824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7801cbe0 con 0x55e756b5a9d0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e6affd640 1 -- 172.21.15.3:0/3270702824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e78021030 con 0x55e756b5a9d0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e6affd640 1 -- 172.21.15.3:0/3270702824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78024370 con 0x55e756b5a9d0 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 -- 172.21.15.3:0/3270702824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 msgr2=0x55e756b5adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2e78013aa0 tx=0x7f2e78013f60 comp rx=0 tx=0).stop 2024-09-12T02:18:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 -- 172.21.15.3:0/3270702824 shutdown_connections 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b67fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b65670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3270702824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b5adb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 -- 172.21.15.3:0/3270702824 >> 172.21.15.3:0/3270702824 conn(0x55e756b56f60 msgr2=0x55e756b57340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 -- 172.21.15.3:0/3270702824 shutdown_connections 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.875+0000 7f2e8921d740 1 -- 172.21.15.3:0/3270702824 wait complete. 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 -- start start 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 Processor -- start 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b5a9d0 0x55e756b65410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e756b68780 con 0x55e756b65bb0 2024-09-12T02:18:09.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e756a7bf20 con 0x55e756b5b2f0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e756b68900 con 0x55e756b5a9d0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e8921d740 10 client.0 _ll_register_callbacks cb 0x55e756a45030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e853e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e853e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60288/0 (socket says 172.21.15.3:60288) 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e853e1640 1 -- 172.21.15.3:0/1859525613 learned_addr learned my addr 172.21.15.3:0/1859525613 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.876+0000 7f2e84be0640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b5a9d0 0x55e756b65410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 -- 172.21.15.3:0/1859525613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b5a9d0 msgr2=0x55e756b65410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b5a9d0 0x55e756b65410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 -- 172.21.15.3:0/1859525613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 msgr2=0x55e756b62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 -- 172.21.15.3:0/1859525613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7568322b0 con 0x55e756b65bb0 2024-09-12T02:18:09.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e853e1640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e780244d0 tx=0x7f2e780134b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e7c399224bd86 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78034060 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2e780135a0 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e78021030 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e7801dd20 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e8921d740 10 client.15129 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e756b74070 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.877+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e78021030 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e7801d510 con 0x55e756b65bb0 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 10 client.15129 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:18:09.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 20 client.15129 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756b76270 0x55e756b776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 4 client.15129 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e756ace600 con 0x55e756b76270 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e84be0640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756b76270 0x55e756b776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e5ffff640 10 client.15129 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.878+0000 7f2e84be0640 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756b76270 0x55e756b776e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=efff4a9682fa5b51 server_cookie=87f43f9fc92b599a in_seq=0 out_seq=0 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e5ffff640 1 -- 172.21.15.3:0/1859525613 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e756ace600 con 0x55e756b76270 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e5ffff640 10 client.15129 handle_command_reply: tid=0 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 client.15129 shutdown 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 msgr2=0x55e756b62ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e780244d0 tx=0x7f2e780134b0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 shutdown_connections 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b65bb0 0x55e756b62ca0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756b76270 0x55e756b776e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e5ffff640 0 client.15129 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 --2- 172.21.15.3:0/1859525613 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b5a9d0 0x55e756b65410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 >> 172.21.15.3:0/1859525613 conn(0x55e756b56f60 msgr2=0x55e756b5cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 shutdown_connections 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 1 -- 172.21.15.3:0/1859525613 wait complete. 2024-09-12T02:18:09.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.879+0000 7f2e8921d740 20 client.15129 trim_cache size 0 max 16384 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 Processor -- start 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 -- start start 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e756b75a20 con 0x55e756b5a9d0 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e756b760f0 con 0x55e756b5b2f0 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e8921d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e756b767c0 con 0x55e756b65bb0 2024-09-12T02:18:09.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e84be0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.897+0000 7f2e84be0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60302/0 (socket says 172.21.15.3:60302) 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 -- 172.21.15.3:0/3021831058 learned_addr learned my addr 172.21.15.3:0/3021831058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 -- 172.21.15.3:0/3021831058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 msgr2=0x55e756b5f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 -- 172.21.15.3:0/3021831058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 msgr2=0x55e756b645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 -- 172.21.15.3:0/3021831058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e756b60cb0 con 0x55e756b5a9d0 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e84be0640 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009610 tx=0x7f2e740bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692fd45c61806c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e6a7fc640 1 -- 172.21.15.3:0/3021831058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7409b330 con 0x55e756b5a9d0 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e6a7fc640 1 -- 172.21.15.3:0/3021831058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7409f6e0 con 0x55e756b5a9d0 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.898+0000 7f2e6a7fc640 1 -- 172.21.15.3:0/3021831058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e740a3d70 con 0x55e756b5a9d0 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- 172.21.15.3:0/3021831058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 msgr2=0x55e756b776e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009610 tx=0x7f2e740bb040 comp rx=0 tx=0).stop 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- 172.21.15.3:0/3021831058 shutdown_connections 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5b2f0 0x55e756b645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 --2- 172.21.15.3:0/3021831058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5a9d0 0x55e756b776e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- 172.21.15.3:0/3021831058 >> 172.21.15.3:0/3021831058 conn(0x55e756b56f60 msgr2=0x55e756b63eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- 172.21.15.3:0/3021831058 shutdown_connections 2024-09-12T02:18:09.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- 172.21.15.3:0/3021831058 wait complete. 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 -- start start 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.899+0000 7f2e8921d740 1 Processor -- start 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5a9d0 0x55e756b7ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e756b77560 con 0x55e756b5b2f0 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e756b76e90 con 0x55e756b5a9d0 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e756b77c30 con 0x55e756b65bb0 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e8921d740 10 client.0 _ll_register_callbacks cb 0x7f2e7409b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60310/0 (socket says 172.21.15.3:60310) 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 -- 172.21.15.3:0/4004988948 learned_addr learned my addr 172.21.15.3:0/4004988948 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e853e1640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5a9d0 0x55e756b7ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 -- 172.21.15.3:0/4004988948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 msgr2=0x55e756b7b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b7b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 -- 172.21.15.3:0/4004988948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5a9d0 msgr2=0x55e756b7ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5a9d0 0x55e756b7ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 -- 172.21.15.3:0/4004988948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e756b7be80 con 0x55e756b5b2f0 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e7ffff640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0314e0 tx=0x7f2e6c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bdd50dc4e613e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.900+0000 7f2e5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:09.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c035030 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2e6c0048c0 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7409f6e0 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e6c005d50 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 10 client.15135 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e756be0820 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e7409f6e0 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2e6c031540 con 0x55e756b5b2f0 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 10 client.15135 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 10 client.15135 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:18:09.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 20 client.15135 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756be53a0 0x55e756be40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 4 client.15135 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "status"}) -- 0x55e756ac8b10 con 0x55e756be53a0 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.901+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756be53a0 0x55e756be40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e84be0640 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756be53a0 0x55e756be40b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f64f67b2d5086188 server_cookie=1d0abbaee3252d07 in_seq=0 out_seq=0 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e5ffff640 10 client.15135 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e5ffff640 1 -- 172.21.15.3:0/4004988948 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e756ac8b10 con 0x55e756be53a0 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e5ffff640 10 client.15135 handle_command_reply: tid=0 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 client.15135 shutdown 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 msgr2=0x55e756b7b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0314e0 tx=0x7f2e6c0052f0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 shutdown_connections 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55e756be53a0 0x55e756be40b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e756b65bb0 0x55e756b7b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e756b5b2f0 0x55e756b7b2a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 --2- 172.21.15.3:0/4004988948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e756b5a9d0 0x55e756b7ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 >> 172.21.15.3:0/4004988948 conn(0x55e756b56f60 msgr2=0x55e756b64660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e5ffff640 0 client.15135 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 shutdown_connections 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.902+0000 7f2e8921d740 1 -- 172.21.15.3:0/4004988948 wait complete. 2024-09-12T02:18:09.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e8921d740 20 client.15135 trim_cache size 0 max 16384 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14619, 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:09.914 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 90.166645826000007, 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 113.59181874799999 2024-09-12T02:18:09.915 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 msgr2=0x7f2e700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 0x7f2e700a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2e78013fd0 tx=0x7f2e7801d000 comp rx=0 tx=0).stop 2024-09-12T02:18:09.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 msgr2=0x7f2e8010f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400d950 tx=0x7f2e7400de10 comp rx=0 tx=0).stop 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 shutdown_connections 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e80102ef0 0x7f2e8010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e80102570 0x7f2e8010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 --2- 172.21.15.3:0/4201274250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e80100e20 0x7f2e8010f4c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 >> 172.21.15.3:0/4201274250 conn(0x7f2e800fc7d0 msgr2=0x7f2e800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 shutdown_connections 2024-09-12T02:18:09.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:09.903+0000 7f2e86e44640 1 -- 172.21.15.3:0/4201274250 wait complete. 2024-09-12T02:18:10.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:10 smithi003 ceph-mon[31476]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:10.673 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:18:10.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:10 smithi026 ceph-mon[35496]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:10.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:10 smithi092 ceph-mon[36089]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:18:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 -- 172.21.15.3:0/715828354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a410c2f0 msgr2=0x7fa9a410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 --2- 172.21.15.3:0/715828354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a410c2f0 0x7fa9a410e790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009ca0 tx=0x7fa99c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 -- 172.21.15.3:0/715828354 shutdown_connections 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 --2- 172.21.15.3:0/715828354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a410c2f0 0x7fa9a410e790 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 --2- 172.21.15.3:0/715828354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a41037a0 0x7fa9a410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 --2- 172.21.15.3:0/715828354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a4102df0 0x7fa9a41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 -- 172.21.15.3:0/715828354 >> 172.21.15.3:0/715828354 conn(0x7fa9a40fc7d0 msgr2=0x7fa9a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 -- 172.21.15.3:0/715828354 shutdown_connections 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.387+0000 7fa9a9909640 1 -- 172.21.15.3:0/715828354 wait complete. 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 Processor -- start 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 -- start start 2024-09-12T02:18:11.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a4102df0 0x7fa9a4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 0x7fa9a419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4112190 con 0x7fa9a41037a0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4112010 con 0x7fa9a410c2f0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a9909640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4112310 con 0x7fa9a4102df0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60322/0 (socket says 172.21.15.3:60322) 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1916649893 learned_addr learned my addr 172.21.15.3:0/1916649893 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a4102df0 0x7fa9a4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 0x7fa9a419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1916649893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a4102df0 msgr2=0x7fa9a4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.388+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a4102df0 0x7fa9a4198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1916649893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 msgr2=0x7fa9a419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 0x7fa9a419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1916649893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a419d600 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 0x7fa9a419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c970 tx=0x7fa99800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a8907640 1 -- 172.21.15.3:0/1916649893 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004220 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a8907640 1 -- 172.21.15.3:0/1916649893 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9980043c0 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a8907640 1 -- 172.21.15.3:0/1916649893 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004a00 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a419d8f0 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.389+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a41a51d0 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.391+0000 7fa9a8907640 1 -- 172.21.15.3:0/1916649893 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa9980079b0 con 0x7fa9a41037a0 2024-09-12T02:18:11.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.391+0000 7fa9abce2740 1 Processor -- start 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 -- start start 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a09891adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098925670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a098881090 con 0x55a098925bb0 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a098828a70 con 0x55a09891b2f0 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9abce2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a09887d050 con 0x55a09891a9d0 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a8907640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 0x7fa98c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a09891adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60330/0 (socket says 172.21.15.3:60330) 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1297455515 learned_addr learned my addr 172.21.15.3:0/1297455515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:11.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098925670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 0x7fa98c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a8907640 1 -- 172.21.15.3:0/1916649893 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9980cedc0 con 0x7fa9a41037a0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1297455515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 msgr2=0x55a09891adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a09891adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1297455515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 msgr2=0x55a098925670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098925670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1297455515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a098835130 con 0x55a098925bb0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.392+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098925670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cc8b0 tx=0x7fa9980053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e495b28eada8454c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 0x7fa98c0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009430 tx=0x7fa99c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa989ffb640 1 -- 172.21.15.3:0/1297455515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99801d070 con 0x55a098925bb0 2024-09-12T02:18:11.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa989ffb640 1 -- 172.21.15.3:0/1297455515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa99809f9c0 con 0x55a098925bb0 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa989ffb640 1 -- 172.21.15.3:0/1297455515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998097c80 con 0x55a098925bb0 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa9abce2740 1 -- 172.21.15.3:0/1297455515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 msgr2=0x55a098927fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.393+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9980cc8b0 tx=0x7fa9980053f0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- 172.21.15.3:0/1297455515 shutdown_connections 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098927fa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098925670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1297455515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a09891adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- 172.21.15.3:0/1297455515 >> 172.21.15.3:0/1297455515 conn(0x55a098916f60 msgr2=0x55a098917340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- 172.21.15.3:0/1297455515 shutdown_connections 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- 172.21.15.3:0/1297455515 wait complete. 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- start start 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 Processor -- start 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891b2f0 0x55a098922830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a098922d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.394+0000 7fa9abce2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a098928780 con 0x55a09891a9d0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a09883bf20 con 0x55a098925bb0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a098928900 con 0x55a09891b2f0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 10 client.0 _ll_register_callbacks cb 0x55a098805030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60334/0 (socket says 172.21.15.3:60334) 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 -- 172.21.15.3:0/1432953900 learned_addr learned my addr 172.21.15.3:0/1432953900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a098922d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891b2f0 0x55a098922830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 -- 172.21.15.3:0/1432953900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891b2f0 msgr2=0x55a098922830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891b2f0 0x55a098922830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 -- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 msgr2=0x55a098922d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a098922d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 -- 172.21.15.3:0/1432953900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0985f22b0 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d440 tx=0x7fa99400d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c386087a1b6cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99401c060 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa994015750 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a09888e600 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa994015950 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 10 client.15147 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.395+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0989341e0 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a09888e600 con 0x55a09891a9d0 2024-09-12T02:18:11.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa9940163e0 con 0x55a09891a9d0 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 10 client.15147 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 20 client.15147 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a098936850 0x55a098937cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 4 client.15147 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:11.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a098888b10 con 0x55a098936850 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.396+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a098936850 0x55a098937cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa982ffd640 10 client.15147 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a098936850 0x55a098937cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b641815434264e0 server_cookie=d489f54e886b7eb1 in_seq=0 out_seq=0 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa982ffd640 1 -- 172.21.15.3:0/1432953900 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a098888b10 con 0x55a098936850 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa982ffd640 10 client.15147 handle_command_reply: tid=0 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9abce2740 1 client.15147 shutdown 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 msgr2=0x55a0989254e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa99400d440 tx=0x7fa99400d900 comp rx=0 tx=0).stop 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 shutdown_connections 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.397+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a098936850 0x55a098937cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a098922d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891b2f0 0x55a098922830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1432953900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891a9d0 0x55a0989254e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 >> 172.21.15.3:0/1432953900 conn(0x55a098916f60 msgr2=0x55a09891cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa982ffd640 0 client.15147 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:11.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 shutdown_connections 2024-09-12T02:18:11.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 1 -- 172.21.15.3:0/1432953900 wait complete. 2024-09-12T02:18:11.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.398+0000 7fa9abce2740 20 client.15147 trim_cache size 0 max 16384 2024-09-12T02:18:11.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 Processor -- start 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 -- start start 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a098936180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a09891d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a098936000 con 0x55a09891b2f0 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a0989366d0 con 0x55a098925bb0 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9abce2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a098936da0 con 0x55a09891a9d0 2024-09-12T02:18:11.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60346/0 (socket says 172.21.15.3:60346) 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1646040829 learned_addr learned my addr 172.21.15.3:0/1646040829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a098936180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a09891d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1646040829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 msgr2=0x55a098936180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a098936180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1646040829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 msgr2=0x55a09891d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a09891d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.416+0000 7fa9a27fc640 1 -- 172.21.15.3:0/1646040829 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a098925100 con 0x55a09891b2f0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa99809b310 tx=0x7fa998086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b9c54b2dab1ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9897fa640 1 -- 172.21.15.3:0/1646040829 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99801d040 con 0x55a09891b2f0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9897fa640 1 -- 172.21.15.3:0/1646040829 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9980b3070 con 0x55a09891b2f0 2024-09-12T02:18:11.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9897fa640 1 -- 172.21.15.3:0/1646040829 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998097c80 con 0x55a09891b2f0 2024-09-12T02:18:11.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 -- 172.21.15.3:0/1646040829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 msgr2=0x55a0989375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa99809b310 tx=0x7fa998086040 comp rx=0 tx=0).stop 2024-09-12T02:18:11.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 -- 172.21.15.3:0/1646040829 shutdown_connections 2024-09-12T02:18:11.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a098925bb0 0x55a09891d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a09891b2f0 0x55a0989375f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 --2- 172.21.15.3:0/1646040829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a098936180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 -- 172.21.15.3:0/1646040829 >> 172.21.15.3:0/1646040829 conn(0x55a098916f60 msgr2=0x55a098921100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 -- 172.21.15.3:0/1646040829 shutdown_connections 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.417+0000 7fa9abce2740 1 -- 172.21.15.3:0/1646040829 wait complete. 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 -- start start 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 Processor -- start 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a0989406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098938dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a098937b40 con 0x55a098925bb0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a098937470 con 0x55a09891b2f0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a098938210 con 0x55a09891a9d0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.418+0000 7fa9abce2740 10 client.0 _ll_register_callbacks cb 0x7fa99400b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60356/0 (socket says 172.21.15.3:60356) 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 -- 172.21.15.3:0/3297815748 learned_addr learned my addr 172.21.15.3:0/3297815748 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a27fc640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098938dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a0989406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 -- 172.21.15.3:0/3297815748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 msgr2=0x55a0989406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a0989406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 -- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 msgr2=0x55a098938dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098938dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 -- 172.21.15.3:0/3297815748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a098939aa0 con 0x55a098925bb0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a0989406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9a37fe640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa99c02ade0 tx=0x7fa99c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d613d3a8070d92a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c051070 con 0x55a098925bb0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa99c004690 con 0x55a098925bb0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9980b3070 con 0x55a098925bb0 2024-09-12T02:18:11.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa99c003e50 con 0x55a098925bb0 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.419+0000 7fa9abce2740 10 client.15153 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a09899ff20 con 0x55a098925bb0 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9980b3070 con 0x55a098925bb0 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa99c030750 con 0x55a098925bb0 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 20 client.15153 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a0989a4ba0 0x55a0989a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 4 client.15153 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:11.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "status"}) -- 0x55a0988050e0 con 0x55a0989a4ba0 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.420+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a0989a4ba0 0x55a0989a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9a2ffd640 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a0989a4ba0 0x55a0989a38b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44673b05990c2cda server_cookie=60a49cbe2bcc344c in_seq=0 out_seq=0 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa982ffd640 10 client.15153 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa982ffd640 1 -- 172.21.15.3:0/3297815748 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a0988050e0 con 0x55a0989a4ba0 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa982ffd640 10 client.15153 handle_command_reply: tid=0 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 client.15153 shutdown 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 msgr2=0x55a098939310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa99c02ade0 tx=0x7fa99c0095c0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 shutdown_connections 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a098925bb0 0x55a098939310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55a0989a4ba0 0x55a0989a38b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a09891b2f0 0x55a098938dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 --2- 172.21.15.3:0/3297815748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a09891a9d0 0x55a0989406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 >> 172.21.15.3:0/3297815748 conn(0x55a098916f60 msgr2=0x55a098916000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa982ffd640 0 client.15153 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 shutdown_connections 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 1 -- 172.21.15.3:0/3297815748 wait complete. 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.421+0000 7fa9abce2740 20 client.15153 trim_cache size 0 max 16384 2024-09-12T02:18:11.429 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24482, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 92.260856997000005, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:11.430 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 112.242991987 2024-09-12T02:18:11.431 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:11.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 msgr2=0x7fa98c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 0x7fa98c0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009430 tx=0x7fa99c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 msgr2=0x7fa9a4198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c970 tx=0x7fa99800ce30 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 shutdown_connections 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa98c0a2b80 0x7fa98c0a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9a410c2f0 0x7fa9a419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9a41037a0 0x7fa9a4198b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 --2- 172.21.15.3:0/1916649893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9a4102df0 0x7fa9a4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 >> 172.21.15.3:0/1916649893 conn(0x7fa9a40fc7d0 msgr2=0x7fa9a40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 shutdown_connections 2024-09-12T02:18:11.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:11.422+0000 7fa9a9909640 1 -- 172.21.15.3:0/1916649893 wait complete. 2024-09-12T02:18:12.168 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:18:12.429 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:12.571 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:12 smithi003 ceph-mon[31476]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:12.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:12 smithi026 ceph-mon[35496]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:12.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:12 smithi092 ceph-mon[36089]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 -- 172.21.15.3:0/2416532980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34100a80 msgr2=0x7f1b34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/2416532980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34100a80 0x7f1b34100ee0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009930 tx=0x7f1b2802a420 comp rx=0 tx=0).stop 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 -- 172.21.15.3:0/2416532980 shutdown_connections 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/2416532980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34101420 0x7f1b3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/2416532980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34100a80 0x7f1b34100ee0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/2416532980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34106a80 0x7f1b34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 -- 172.21.15.3:0/2416532980 >> 172.21.15.3:0/2416532980 conn(0x7f1b340fc7a0 msgr2=0x7f1b340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 -- 172.21.15.3:0/2416532980 shutdown_connections 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.905+0000 7f1b38b7e640 1 -- 172.21.15.3:0/2416532980 wait complete. 2024-09-12T02:18:12.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 Processor -- start 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 -- start start 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34100a80 0x7f1b341974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34106a80 0x7f1b3419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110f20 con 0x7f1b34101420 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1b34110da0 con 0x7f1b34106a80 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b38b7e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1b341110a0 con 0x7f1b34100a80 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60372/0 (socket says 172.21.15.3:60372) 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.906+0000 7f1b31d9b640 1 -- 172.21.15.3:0/3561787454 learned_addr learned my addr 172.21.15.3:0/3561787454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34106a80 0x7f1b3419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b3259c640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34100a80 0x7f1b341974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 -- 172.21.15.3:0/3561787454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34100a80 msgr2=0x7f1b341974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34100a80 0x7f1b341974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 -- 172.21.15.3:0/3561787454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34106a80 msgr2=0x7f1b3419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34106a80 0x7f1b3419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 -- 172.21.15.3:0/3561787454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3419c4a0 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009a50 tx=0x7f1b28004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b237fe640 1 -- 172.21.15.3:0/3561787454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2803f030 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b237fe640 1 -- 172.21.15.3:0/3561787454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b28003ce0 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3419c730 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b237fe640 1 -- 172.21.15.3:0/3561787454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28035070 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.907+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b341a4070 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.909+0000 7f1b237fe640 1 -- 172.21.15.3:0/3561787454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1b28042030 con 0x7f1b34101420 2024-09-12T02:18:12.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 Processor -- start 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 -- start start 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7989870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d798a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5571d78f0090 con 0x5571d7998c70 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d7897a70 con 0x5571d7989e40 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b3af57740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571d78ec050 con 0x5571d798b070 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b237fe640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 0x7f1b1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.910+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 0x7f1b1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60378/0 (socket says 172.21.15.3:60378) 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 -- 172.21.15.3:0/1619860940 learned_addr learned my addr 172.21.15.3:0/1619860940 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:12.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b3259c640 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d798a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7989870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b237fe640 1 -- 172.21.15.3:0/3561787454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1b280ef620 con 0x7f1b34101420 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 -- 172.21.15.3:0/1619860940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 msgr2=0x5571d7989870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7989870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 -- 172.21.15.3:0/1619860940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 msgr2=0x5571d798a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d798a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 -- 172.21.15.3:0/1619860940 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d78a4130 con 0x5571d7998c70 2024-09-12T02:18:12.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 0x7f1b1c0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00abb0 tx=0x7f1b2c00a6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b28031550 tx=0x7f1b2803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580ae23204e6595c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b20ff9640 1 -- 172.21.15.3:0/1619860940 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280bc570 con 0x5571d7998c70 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b20ff9640 1 -- 172.21.15.3:0/1619860940 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b280bbe70 con 0x5571d7998c70 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.911+0000 7f1b20ff9640 1 -- 172.21.15.3:0/1619860940 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280b84e0 con 0x5571d7998c70 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- 172.21.15.3:0/1619860940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 msgr2=0x5571d799b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b28031550 tx=0x7f1b2803b800 comp rx=0 tx=0).stop 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- 172.21.15.3:0/1619860940 shutdown_connections 2024-09-12T02:18:12.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d799b060 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d798a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1619860940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7989870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- 172.21.15.3:0/1619860940 >> 172.21.15.3:0/1619860940 conn(0x5571d7985f60 msgr2=0x5571d7986340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- 172.21.15.3:0/1619860940 shutdown_connections 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- 172.21.15.3:0/1619860940 wait complete. 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.912+0000 7f1b3af57740 1 -- start start 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 Processor -- start 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d79985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7995940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5571d799b920 con 0x5571d7998c70 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d78aaf20 con 0x5571d7989e40 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571d799baa0 con 0x5571d798b070 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3af57740 10 client.0 _ll_register_callbacks cb 0x5571d7874030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60388/0 (socket says 172.21.15.3:60388) 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 -- 172.21.15.3:0/1935687812 learned_addr learned my addr 172.21.15.3:0/1935687812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:12.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7995940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3259c640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d79985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 -- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 msgr2=0x5571d7995940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7995940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 -- 172.21.15.3:0/1935687812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 msgr2=0x5571d79985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d79985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b32d9d640 1 -- 172.21.15.3:0/1935687812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d76612b0 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.913+0000 7f1b3259c640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d79985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c016f60 tx=0x7f1b2c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c834c334e3183c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c028060 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1b2c019e00 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b280bbe70 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c018560 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 10 client.15165 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571d79a7240 con 0x5571d7998c70 2024-09-12T02:18:12.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b280bbe70 con 0x5571d7998c70 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b2c022070 con 0x5571d7998c70 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 10 client.15165 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 10 client.15165 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 20 client.15165 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d79a9880 0x5571d79aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 4 client.15165 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.914+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d78fd600 con 0x5571d79a9880 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.915+0000 7f1b3259c640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d79a9880 0x5571d79aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.915+0000 7f1b19ffb640 10 client.15165 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.915+0000 7f1b3259c640 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d79a9880 0x5571d79aacf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dda65647303efb04 server_cookie=7cc5d3ebcca2ce73 in_seq=0 out_seq=0 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b19ffb640 1 -- 172.21.15.3:0/1935687812 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571d78fd600 con 0x5571d79a9880 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b19ffb640 10 client.15165 handle_command_reply: tid=0 2024-09-12T02:18:12.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 client.15165 shutdown 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 msgr2=0x5571d7995ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c016f60 tx=0x7f1b2c0093d0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 shutdown_connections 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7998c70 0x5571d7995ec0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d79a9880 0x5571d79aacf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d798b070 0x5571d7995940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 --2- 172.21.15.3:0/1935687812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7989e40 0x5571d79985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b19ffb640 0 client.15165 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 >> 172.21.15.3:0/1935687812 conn(0x5571d7985f60 msgr2=0x5571d798bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 shutdown_connections 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 1 -- 172.21.15.3:0/1935687812 wait complete. 2024-09-12T02:18:12.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.916+0000 7f1b3af57740 20 client.15165 trim_cache size 0 max 16384 2024-09-12T02:18:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 Processor -- start 2024-09-12T02:18:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 -- start start 2024-09-12T02:18:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 0x5571d79a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7998c70 0x5571d7995900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5571d79a9030 con 0x5571d7989e40 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d79a9700 con 0x5571d798b070 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.934+0000 7f1b3af57740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571d79a9dd0 con 0x5571d7998c70 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60392/0 (socket says 172.21.15.3:60392) 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 -- 172.21.15.3:0/2838172324 learned_addr learned my addr 172.21.15.3:0/2838172324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7998c70 0x5571d7995900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 0x5571d79a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 -- 172.21.15.3:0/2838172324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7998c70 msgr2=0x5571d7995900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7998c70 0x5571d7995900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 -- 172.21.15.3:0/2838172324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 msgr2=0x5571d79a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 0x5571d79a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 -- 172.21.15.3:0/2838172324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d7993e90 con 0x5571d7989e40 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 0x5571d79a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b3259c640 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5571d79ab7d0 tx=0x7f1b24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c3d2498b370dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b1bfff640 1 -- 172.21.15.3:0/2838172324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24016b20 con 0x5571d7989e40 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b1bfff640 1 -- 172.21.15.3:0/2838172324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b24020030 con 0x5571d7989e40 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.935+0000 7f1b1bfff640 1 -- 172.21.15.3:0/2838172324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2401c250 con 0x5571d7989e40 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- 172.21.15.3:0/2838172324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 msgr2=0x5571d7989750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5571d79ab7d0 tx=0x7f1b24008040 comp rx=0 tx=0).stop 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- 172.21.15.3:0/2838172324 shutdown_connections 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7998c70 0x5571d7995900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d798b070 0x5571d79a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 --2- 172.21.15.3:0/2838172324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d7989e40 0x5571d7989750 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- 172.21.15.3:0/2838172324 >> 172.21.15.3:0/2838172324 conn(0x5571d7985f60 msgr2=0x5571d7994e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- 172.21.15.3:0/2838172324 shutdown_connections 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- 172.21.15.3:0/2838172324 wait complete. 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 -- start start 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.936+0000 7f1b3af57740 1 Processor -- start 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7989e40 0x5571d79b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7998c70 0x5571d79a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5571d79aab70 con 0x5571d798b070 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5571d79aa4a0 con 0x5571d7998c70 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5571d79ab240 con 0x5571d7989e40 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3af57740 10 client.0 _ll_register_callbacks cb 0x7f1b2c016c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60402/0 (socket says 172.21.15.3:60402) 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 -- 172.21.15.3:0/4266977864 learned_addr learned my addr 172.21.15.3:0/4266977864 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b32d9d640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7998c70 0x5571d79a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b3259c640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7989e40 0x5571d79b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 -- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7989e40 msgr2=0x5571d79b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7989e40 0x5571d79b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 -- 172.21.15.3:0/4266977864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7998c70 msgr2=0x5571d79a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7998c70 0x5571d79a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 -- 172.21.15.3:0/4266977864 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d79a4cf0 con 0x5571d798b070 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b31d9b640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b280ed480 tx=0x7f1b2802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950318b3a88be62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.937+0000 7f1b19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28049040 con 0x5571d798b070 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1b280b84e0 con 0x5571d798b070 2024-09-12T02:18:12.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b24020030 con 0x5571d798b070 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1b280bc570 con 0x5571d798b070 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 10 client.15171 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571d7a13f90 con 0x5571d798b070 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b24020030 con 0x5571d798b070 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1b2809ace0 con 0x5571d798b070 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 10 client.15171 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 10 client.15171 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 20 client.15171 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d7a185b0 0x5571d7a172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 4 client.15171 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.938+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "status"}) -- 0x5571d78f7b10 con 0x5571d7a185b0 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3259c640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d7a185b0 0x5571d7a172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:12.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3259c640 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d7a185b0 0x5571d7a172c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da9b54cf2d502335 server_cookie=fbd9cd68c115efd3 in_seq=0 out_seq=0 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b19ffb640 10 client.15171 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b19ffb640 1 -- 172.21.15.3:0/4266977864 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5571d78f7b10 con 0x5571d7a185b0 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b19ffb640 10 client.15171 handle_command_reply: tid=0 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 client.15171 shutdown 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 msgr2=0x5571d79a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b280ed480 tx=0x7f1b2802f000 comp rx=0 tx=0).stop 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 shutdown_connections 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5571d7998c70 0x5571d79a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5571d798b070 0x5571d79a4070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x5571d7a185b0 0x5571d7a172c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 --2- 172.21.15.3:0/4266977864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5571d7989e40 0x5571d79b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 >> 172.21.15.3:0/4266977864 conn(0x5571d7985f60 msgr2=0x5571d7995900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b19ffb640 0 client.15171 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.939+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 shutdown_connections 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b3af57740 1 -- 172.21.15.3:0/4266977864 wait complete. 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b3af57740 20 client.15171 trim_cache size 0 max 16384 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24487, 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:12.957 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 94.791610820000002, 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 130.76685147399999 2024-09-12T02:18:12.958 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 msgr2=0x7f1b1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 0x7f1b1c0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00abb0 tx=0x7f1b2c00a6a0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 msgr2=0x7f1b34197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009a50 tx=0x7f1b28004270 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 shutdown_connections 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1b1c0a2c00 0x7f1b1c0a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b34106a80 0x7f1b3419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1b34101420 0x7f1b34197a30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 --2- 172.21.15.3:0/3561787454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b34100a80 0x7f1b341974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 >> 172.21.15.3:0/3561787454 conn(0x7f1b340fc7a0 msgr2=0x7f1b340fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 shutdown_connections 2024-09-12T02:18:12.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:12.940+0000 7f1b38b7e640 1 -- 172.21.15.3:0/3561787454 wait complete. 2024-09-12T02:18:13.711 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:18:13.970 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:14.377 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:14 smithi003 ceph-mon[31476]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/3074909076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 msgr2=0x7f9a1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/3074909076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a1810ea90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009ca0 tx=0x7f9a1002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/3074909076 shutdown_connections 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/3074909076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a1810ea90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/3074909076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18102790 0x7f9a181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/3074909076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18101de0 0x7f9a181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.456+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/3074909076 >> 172.21.15.3:0/3074909076 conn(0x7f9a180fd8c0 msgr2=0x7f9a180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.457+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/3074909076 shutdown_connections 2024-09-12T02:18:14.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.457+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/3074909076 wait complete. 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.457+0000 7f9a1e6a0640 1 Processor -- start 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.457+0000 7f9a1e6a0640 1 -- start start 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18101de0 0x7f9a180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18102790 0x7f9a18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111f90 con 0x7f9a18106f10 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18111e10 con 0x7f9a18102790 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18112110 con 0x7f9a18101de0 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60432/0 (socket says 172.21.15.3:60432) 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 -- 172.21.15.3:0/1722656969 learned_addr learned my addr 172.21.15.3:0/1722656969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:14.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a17fff640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18101de0 0x7f9a180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a177fe640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18102790 0x7f9a18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 -- 172.21.15.3:0/1722656969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18101de0 msgr2=0x7f9a180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18101de0 0x7f9a180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 -- 172.21.15.3:0/1722656969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18102790 msgr2=0x7f9a18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18102790 0x7f9a18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 -- 172.21.15.3:0/1722656969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a181a6910 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1cc3d640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a10031a60 tx=0x7f9a10031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a157fa640 1 -- 172.21.15.3:0/1722656969 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10003c30 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a157fa640 1 -- 172.21.15.3:0/1722656969 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a10003dd0 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a181a6ba0 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.458+0000 7f9a157fa640 1 -- 172.21.15.3:0/1722656969 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a100047e0 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.459+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a7130 con 0x7f9a18106f10 2024-09-12T02:18:14.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.461+0000 7f9a157fa640 1 -- 172.21.15.3:0/1722656969 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9a1002f790 con 0x7f9a18106f10 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.461+0000 7f9a20a79740 1 Processor -- start 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.461+0000 7f9a20a79740 1 -- start start 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364bcbb0 0x55d6364befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d636418090 con 0x55d6364b19d0 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d6363bfa70 con 0x55d6364b22f0 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a20a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d636414050 con 0x55d6364bcbb0 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a157fa640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60448/0 (socket says 172.21.15.3:60448) 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 -- 172.21.15.3:0/292922520 learned_addr learned my addr 172.21.15.3:0/292922520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:14.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364bcbb0 0x55d6364befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a1cc3d640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a17fff640 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 -- 172.21.15.3:0/292922520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364bcbb0 msgr2=0x55d6364befa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364bcbb0 0x55d6364befa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a157fa640 1 -- 172.21.15.3:0/1722656969 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9a100f03c0 con 0x7f9a18106f10 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 -- 172.21.15.3:0/292922520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 msgr2=0x55d6364bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 -- 172.21.15.3:0/292922520 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6363cc130 con 0x55d6364b19d0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.462+0000 7f9a177fe640 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55d6364af2e0 tx=0x7f9a0c009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0cd1f12124042e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a1cc3d640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 0x7f9a000a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9a100bca30 tx=0x7f9a100e77a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f99fe7fc640 1 -- 172.21.15.3:0/292922520 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01d070 con 0x55d6364b19d0 2024-09-12T02:18:14.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f99fe7fc640 1 -- 172.21.15.3:0/292922520 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a0c010040 con 0x55d6364b19d0 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f99fe7fc640 1 -- 172.21.15.3:0/292922520 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00d030 con 0x55d6364b19d0 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 -- 172.21.15.3:0/292922520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 msgr2=0x55d6364b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55d6364af2e0 tx=0x7f9a0c009420 comp rx=0 tx=0).stop 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 -- 172.21.15.3:0/292922520 shutdown_connections 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364bcbb0 0x55d6364befa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 --2- 172.21.15.3:0/292922520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b19d0 0x55d6364b1db0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 -- 172.21.15.3:0/292922520 >> 172.21.15.3:0/292922520 conn(0x55d6364adf60 msgr2=0x55d6364ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 -- 172.21.15.3:0/292922520 shutdown_connections 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.463+0000 7f9a20a79740 1 -- 172.21.15.3:0/292922520 wait complete. 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 -- start start 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 Processor -- start 2024-09-12T02:18:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b19d0 0x55d6364bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b22f0 0x55d6364b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d6364bf780 con 0x55d6364bcbb0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d6363d2f20 con 0x55d6364b19d0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6364bf900 con 0x55d6364b22f0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a20a79740 10 client.0 _ll_register_callbacks cb 0x55d63639c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60462/0 (socket says 172.21.15.3:60462) 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.464+0000 7f9a177fe640 1 -- 172.21.15.3:0/3846406337 learned_addr learned my addr 172.21.15.3:0/3846406337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a17fff640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b22f0 0x55d6364b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b19d0 0x55d6364bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 -- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b22f0 msgr2=0x55d6364b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b22f0 0x55d6364b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 -- 172.21.15.3:0/3846406337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b19d0 msgr2=0x55d6364bc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b19d0 0x55d6364bc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 -- 172.21.15.3:0/3846406337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6361892b0 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a177fe640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0099a0 tx=0x7f9a0c018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b697de4c7ecac18 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f99f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c00d070 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9a0c005bc0 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0c010040 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c01d040 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a20a79740 10 client.15183 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6364cb120 con 0x55d6364bcbb0 2024-09-12T02:18:14.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.465+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a0c010040 con 0x55d6364bcbb0 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a0c0145a0 con 0x55d6364bcbb0 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 10 client.15183 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 10 client.15183 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 20 client.15183 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d6364ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 4 client.15183 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d636425600 con 0x55d6364cd250 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a17fff640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d6364ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f99f77fe640 10 client.15183 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.466+0000 7f9a17fff640 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d6364ce6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f654372e4cbc749 server_cookie=2e52f33ad67232df in_seq=0 out_seq=0 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f99f77fe640 1 -- 172.21.15.3:0/3846406337 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d636425600 con 0x55d6364cd250 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f99f77fe640 10 client.15183 handle_command_reply: tid=0 2024-09-12T02:18:14.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 client.15183 shutdown 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 msgr2=0x55d6364b9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0099a0 tx=0x7f9a0c018640 comp rx=0 tx=0).stop 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 shutdown_connections 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d6364ce6c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364b9d30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f99f77fe640 0 client.15183 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b22f0 0x55d6364b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 --2- 172.21.15.3:0/3846406337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b19d0 0x55d6364bc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.467+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 >> 172.21.15.3:0/3846406337 conn(0x55d6364adf60 msgr2=0x55d6364b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.468+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 shutdown_connections 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.468+0000 7f9a20a79740 1 -- 172.21.15.3:0/3846406337 wait complete. 2024-09-12T02:18:14.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.468+0000 7f9a20a79740 20 client.15183 trim_cache size 0 max 16384 2024-09-12T02:18:14.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.485+0000 7f9a20a79740 1 Processor -- start 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 -- start start 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 0x55d6364b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d6364cca00 con 0x55d6364b22f0 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d6364cd0d0 con 0x55d6364bcbb0 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a20a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6364cd7a0 con 0x55d6364b19d0 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60478/0 (socket says 172.21.15.3:60478) 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 -- 172.21.15.3:0/489245583 learned_addr learned my addr 172.21.15.3:0/489245583 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a177fe640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 0x55d6364b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a177fe640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 -- 172.21.15.3:0/489245583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 msgr2=0x55d6364ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 -- 172.21.15.3:0/489245583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 msgr2=0x55d6364b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 0x55d6364b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 -- 172.21.15.3:0/489245583 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6364bc810 con 0x55d6364b22f0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a177fe640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 0x55d6364b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.486+0000 7f9a17fff640 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005d90 tx=0x7f9a08013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703d4d8350646de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f99fdffb640 1 -- 172.21.15.3:0/489245583 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08004020 con 0x55d6364b22f0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f99fdffb640 1 -- 172.21.15.3:0/489245583 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a080041c0 con 0x55d6364b22f0 2024-09-12T02:18:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f99fdffb640 1 -- 172.21.15.3:0/489245583 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0801d690 con 0x55d6364b22f0 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 -- 172.21.15.3:0/489245583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 msgr2=0x55d6364b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005d90 tx=0x7f9a08013830 comp rx=0 tx=0).stop 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 -- 172.21.15.3:0/489245583 shutdown_connections 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364bcbb0 0x55d6364b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364b22f0 0x55d6364b80c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 --2- 172.21.15.3:0/489245583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 -- 172.21.15.3:0/489245583 >> 172.21.15.3:0/489245583 conn(0x55d6364adf60 msgr2=0x55d6364a8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 -- 172.21.15.3:0/489245583 shutdown_connections 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.487+0000 7f9a20a79740 1 -- 172.21.15.3:0/489245583 wait complete. 2024-09-12T02:18:14.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 -- start start 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 Processor -- start 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d6364ce540 con 0x55d6364bcbb0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d6364cde70 con 0x55d6364b22f0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d6364cec10 con 0x55d6364b19d0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a20a79740 10 client.0 _ll_register_callbacks cb 0x7f9a0c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60490/0 (socket says 172.21.15.3:60490) 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.488+0000 7f9a177fe640 1 -- 172.21.15.3:0/1599965637 learned_addr learned my addr 172.21.15.3:0/1599965637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a17fff640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 -- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 msgr2=0x55d6364d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364d6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 -- 172.21.15.3:0/1599965637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 msgr2=0x55d6364d6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364d6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 -- 172.21.15.3:0/1599965637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6364d0550 con 0x55d6364bcbb0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a17fff640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a177fe640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c016550 tx=0x7f9a0c018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77446a627c38ea8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f99f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c033060 con 0x55d6364bcbb0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9a0c014be0 con 0x55d6364bcbb0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a080041c0 con 0x55d6364bcbb0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a0c00d040 con 0x55d6364bcbb0 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a20a79740 10 client.15189 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:14.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6364d4ee0 con 0x55d6364bcbb0 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.489+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a080041c0 con 0x55d6364bcbb0 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9a0c02f470 con 0x55d6364bcbb0 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 10 client.15189 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 20 client.15189 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d63653ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 4 client.15189 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "status"}) -- 0x55d63641fb10 con 0x55d6364cd250 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a17fff640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d63653ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f99f77fe640 10 client.15189 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.490+0000 7f9a17fff640 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d63653ab40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13a5c7f0033bf12e server_cookie=2a12d6ebcd6f1ca in_seq=0 out_seq=0 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f99f77fe640 1 -- 172.21.15.3:0/1599965637 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55d63641fb10 con 0x55d6364cd250 2024-09-12T02:18:14.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f99f77fe640 10 client.15189 handle_command_reply: tid=0 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 client.15189 shutdown 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 msgr2=0x55d6364cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c016550 tx=0x7f9a0c018640 comp rx=0 tx=0).stop 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 shutdown_connections 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d6364cd250 0x55d63653ab40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d6364bcbb0 0x55d6364cfe60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6364b22f0 0x55d6364d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 --2- 172.21.15.3:0/1599965637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d6364b19d0 0x55d6364d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 >> 172.21.15.3:0/1599965637 conn(0x55d6364adf60 msgr2=0x55d6364b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f99f77fe640 0 client.15189 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 shutdown_connections 2024-09-12T02:18:14.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 1 -- 172.21.15.3:0/1599965637 wait complete. 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.491+0000 7f9a20a79740 20 client.15189 trim_cache size 0 max 16384 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 97.350341555, 2024-09-12T02:18:14.498 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 129.577774097 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 msgr2=0x7f9a000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 0x7f9a000a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9a100bca30 tx=0x7f9a100e77a0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 msgr2=0x7f9a18074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:14.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a10031a60 tx=0x7f9a10031a90 comp rx=0 tx=0).stop 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 shutdown_connections 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9a000a2c00 0x7f9a000a50c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a18106f10 0x7f9a18074f70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18102790 0x7f9a18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 --2- 172.21.15.3:0/1722656969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9a18101de0 0x7f9a180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 >> 172.21.15.3:0/1722656969 conn(0x7f9a180fd8c0 msgr2=0x7f9a180fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 shutdown_connections 2024-09-12T02:18:14.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:14.492+0000 7f9a1e6a0640 1 -- 172.21.15.3:0/1722656969 wait complete. 2024-09-12T02:18:14.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:14 smithi026 ceph-mon[35496]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-12T02:18:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:14 smithi092 ceph-mon[36089]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-12T02:18:15.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:18:15.890 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- 172.21.15.3:0/2466630481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102ef0 msgr2=0x7fa71810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2466630481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102ef0 0x7fa71810da40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ca0 tx=0x7fa71002a950 comp rx=0 tx=0).stop 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- 172.21.15.3:0/2466630481 shutdown_connections 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2466630481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102ef0 0x7fa71810da40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2466630481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718102570 0x7fa7181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2466630481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718100e20 0x7fa718101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- 172.21.15.3:0/2466630481 >> 172.21.15.3:0/2466630481 conn(0x7fa7180fc7d0 msgr2=0x7fa7180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- 172.21.15.3:0/2466630481 shutdown_connections 2024-09-12T02:18:16.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- 172.21.15.3:0/2466630481 wait complete. 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 Processor -- start 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.400+0000 7fa71fda9640 1 -- start start 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718100e20 0x7fa7181984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718102ef0 0x7fa71819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa718110fd0 con 0x7fa718102570 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa718110e50 con 0x7fa718102ef0 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71fda9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa718111150 con 0x7fa718100e20 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60506/0 (socket says 172.21.15.3:60506) 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 -- 172.21.15.3:0/2143220795 learned_addr learned my addr 172.21.15.3:0/2143220795 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:16.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71db45640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718100e20 0x7fa7181984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71e346640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718102ef0 0x7fa71819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 -- 172.21.15.3:0/2143220795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718100e20 msgr2=0x7fa7181984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718100e20 0x7fa7181984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 -- 172.21.15.3:0/2143220795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718102ef0 msgr2=0x7fa71819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718102ef0 0x7fa71819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 -- 172.21.15.3:0/2143220795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71819d470 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa71d344640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c970 tx=0x7fa70800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.401+0000 7fa70effd640 1 -- 172.21.15.3:0/2143220795 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708007bb0 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.402+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71819d760 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.402+0000 7fa70effd640 1 -- 172.21.15.3:0/2143220795 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708007d50 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.402+0000 7fa70effd640 1 -- 172.21.15.3:0/2143220795 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70800f450 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.402+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a5040 con 0x7fa718102570 2024-09-12T02:18:16.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.404+0000 7fa70effd640 1 -- 172.21.15.3:0/2143220795 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa708005020 con 0x7fa718102570 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.404+0000 7fa722182740 1 Processor -- start 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.404+0000 7fa722182740 1 -- start start 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa70effd640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 0x7fa7040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e37670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a766d93090 con 0x55a766e37bb0 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a766d3aa70 con 0x55a766e2c9d0 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa722182740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a766d8f050 con 0x55a766e2d2f0 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71e346640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 0x7fa7040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:42380/0 (socket says 172.21.15.3:42380) 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa70effd640 1 -- 172.21.15.3:0/2143220795 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa7080ced80 con 0x7fa718102570 2024-09-12T02:18:16.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 -- 172.21.15.3:0/1653161109 learned_addr learned my addr 172.21.15.3:0/1653161109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71d344640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e37670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71e346640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 -- 172.21.15.3:0/1653161109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 msgr2=0x55a766e37670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e37670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 -- 172.21.15.3:0/1653161109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 msgr2=0x55a766e39fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e39fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 -- 172.21.15.3:0/1653161109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a766d47130 con 0x55a766e2c9d0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71d344640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e37670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71e346640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 0x7fa7040a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa710006010 tx=0x7fa710005f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.405+0000 7fa71db45640 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa71400bd10 tx=0x7fa7140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152344f3a87ae038 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa703fff640 1 -- 172.21.15.3:0/1653161109 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714004030 con 0x55a766e2c9d0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa703fff640 1 -- 172.21.15.3:0/1653161109 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7140041d0 con 0x55a766e2c9d0 2024-09-12T02:18:16.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa703fff640 1 -- 172.21.15.3:0/1653161109 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714011840 con 0x55a766e2c9d0 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 -- 172.21.15.3:0/1653161109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 msgr2=0x55a766e2cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa71400bd10 tx=0x7fa7140107a0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 -- 172.21.15.3:0/1653161109 shutdown_connections 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e39fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e37670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 --2- 172.21.15.3:0/1653161109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e2cdb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.406+0000 7fa722182740 1 -- 172.21.15.3:0/1653161109 >> 172.21.15.3:0/1653161109 conn(0x55a766e28f60 msgr2=0x55a766e29340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 -- 172.21.15.3:0/1653161109 shutdown_connections 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 -- 172.21.15.3:0/1653161109 wait complete. 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 -- start start 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 Processor -- start 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.407+0000 7fa722182740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a766e3a780 con 0x55a766e37bb0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a766d4df20 con 0x55a766e2c9d0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a766e3a900 con 0x55a766e2d2f0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 10 client.0 _ll_register_callbacks cb 0x55a766d17030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60530/0 (socket says 172.21.15.3:60530) 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71e346640 1 -- 172.21.15.3:0/1252786676 learned_addr learned my addr 172.21.15.3:0/1252786676 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71d344640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 -- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 msgr2=0x55a766e347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 -- 172.21.15.3:0/1252786676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 msgr2=0x55a766e34d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 -- 172.21.15.3:0/1252786676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a766b042b0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71e346640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa71db45640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e37470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa71400ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e9e7f7cf50e3231 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa7017fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714029060 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7140041d0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.408+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa714004380 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7140035f0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 10 client.24854 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a766e460d0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7140041d0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa7140046c0 con 0x55a766e2c9d0 2024-09-12T02:18:16.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 10 client.24854 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 10 client.24854 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 20 client.24854 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766e48310 0x55a766e49780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 4 client.24854 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.409+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a766da0600 con 0x55a766e48310 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.410+0000 7fa71d344640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766e48310 0x55a766e49780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.410+0000 7fa71d344640 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766e48310 0x55a766e49780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0f0041537a668d server_cookie=60acf81a74f6b0ec in_seq=0 out_seq=0 2024-09-12T02:18:16.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.410+0000 7fa7017fa640 10 client.24854 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa7017fa640 1 -- 172.21.15.3:0/1252786676 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a766da0600 con 0x55a766e48310 2024-09-12T02:18:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa7017fa640 10 client.24854 handle_command_reply: tid=0 2024-09-12T02:18:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 client.24854 shutdown 2024-09-12T02:18:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 msgr2=0x55a766e37470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e37470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa71400ba20 comp rx=0 tx=0).stop 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 shutdown_connections 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766e48310 0x55a766e49780 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e34d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa7017fa640 0 client.24854 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2d2f0 0x55a766e347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 --2- 172.21.15.3:0/1252786676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2c9d0 0x55a766e37470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 >> 172.21.15.3:0/1252786676 conn(0x55a766e28f60 msgr2=0x55a766e2ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 shutdown_connections 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 1 -- 172.21.15.3:0/1252786676 wait complete. 2024-09-12T02:18:16.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.411+0000 7fa722182740 20 client.24854 trim_cache size 0 max 16384 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 Processor -- start 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 -- start start 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e49780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a766e47ac0 con 0x55a766e37bb0 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a766e48190 con 0x55a766e2d2f0 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa722182740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a766e48860 con 0x55a766e2c9d0 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60544/0 (socket says 172.21.15.3:60544) 2024-09-12T02:18:16.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa71e346640 1 -- 172.21.15.3:0/965109338 learned_addr learned my addr 172.21.15.3:0/965109338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.429+0000 7fa71db45640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e49780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71d344640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 -- 172.21.15.3:0/965109338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 msgr2=0x55a766e49780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e49780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 -- 172.21.15.3:0/965109338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 msgr2=0x55a766e27ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e27ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 -- 172.21.15.3:0/965109338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a766e3a890 con 0x55a766e37bb0 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71db45640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e49780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:16.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa71e346640 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa71002ae70 tx=0x7fa71002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa0424d54434d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa7037fe640 1 -- 172.21.15.3:0/965109338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71003e040 con 0x55a766e37bb0 2024-09-12T02:18:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa7037fe640 1 -- 172.21.15.3:0/965109338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa710044040 con 0x55a766e37bb0 2024-09-12T02:18:16.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.430+0000 7fa7037fe640 1 -- 172.21.15.3:0/965109338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710034040 con 0x55a766e37bb0 2024-09-12T02:18:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- 172.21.15.3:0/965109338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 msgr2=0x55a766e31a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa71002ae70 tx=0x7fa71002a950 comp rx=0 tx=0).stop 2024-09-12T02:18:16.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- 172.21.15.3:0/965109338 shutdown_connections 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766e31a20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e27ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 --2- 172.21.15.3:0/965109338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e49780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- 172.21.15.3:0/965109338 >> 172.21.15.3:0/965109338 conn(0x55a766e28f60 msgr2=0x55a766e277e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- 172.21.15.3:0/965109338 shutdown_connections 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- 172.21.15.3:0/965109338 wait complete. 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 -- start start 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.431+0000 7fa722182740 1 Processor -- start 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a766e49600 con 0x55a766e37bb0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a766e48f30 con 0x55a766e2d2f0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a766e49cd0 con 0x55a766e2c9d0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 10 client.0 _ll_register_callbacks cb 0x7fa708005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60556/0 (socket says 172.21.15.3:60556) 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 -- 172.21.15.3:0/2922422050 learned_addr learned my addr 172.21.15.3:0/2922422050 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71db45640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71d344640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 -- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 msgr2=0x55a766e4cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e4cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 -- 172.21.15.3:0/2922422050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 msgr2=0x55a766e4d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e4d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 -- 172.21.15.3:0/2922422050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a766eb1000 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa71e346640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ca0 tx=0x7fa71002f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eefaefd77db080 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa7017fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71003e040 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa71003a430 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.432+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa710044040 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa710030da0 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 10 client.15207 fetch_fsmap learned FSMap version 28 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a766eb22e0 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa710044040 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa710060400 con 0x55a766e37bb0 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 10 client.15207 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T02:18:16.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 10 client.15207 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 20 client.15207 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766eb6a90 0x55a766eb57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 4 client.15207 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "status"}) -- 0x55a766d9ab10 con 0x55a766eb6a90 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.433+0000 7fa71db45640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766eb6a90 0x55a766eb57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa7017fa640 10 client.15207 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa71db45640 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766eb6a90 0x55a766eb57a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e209c8ac5887bfb6 server_cookie=febd6a0d9009a97c in_seq=0 out_seq=0 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa7017fa640 1 -- 172.21.15.3:0/2922422050 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55a766d9ab10 con 0x55a766eb6a90 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa7017fa640 10 client.15207 handle_command_reply: tid=0 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 client.15207 shutdown 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 msgr2=0x55a766eb0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa710009ca0 tx=0x7fa71002f4d0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 shutdown_connections 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a766eb6a90 0x55a766eb57a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a766e37bb0 0x55a766eb0a10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a766e2d2f0 0x55a766e4d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 --2- 172.21.15.3:0/2922422050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a766e2c9d0 0x55a766e4cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa7017fa640 0 client.15207 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 >> 172.21.15.3:0/2922422050 conn(0x55a766e28f60 msgr2=0x55a766e36690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 shutdown_connections 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 1 -- 172.21.15.3:0/2922422050 wait complete. 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.434+0000 7fa722182740 20 client.15207 trim_cache size 0 max 16384 2024-09-12T02:18:16.441 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24508, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 106.31896748, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 28, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 125.906987149 2024-09-12T02:18:16.442 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 msgr2=0x7fa7040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 0x7fa7040a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa710006010 tx=0x7fa710005f10 comp rx=0 tx=0).stop 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 msgr2=0x7fa718198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c970 tx=0x7fa70800ce30 comp rx=0 tx=0).stop 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 shutdown_connections 2024-09-12T02:18:16.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa7040a2b80 0x7fa7040a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa718102ef0 0x7fa71819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa718102570 0x7fa718198a00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 --2- 172.21.15.3:0/2143220795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa718100e20 0x7fa7181984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 >> 172.21.15.3:0/2143220795 conn(0x7fa7180fc7d0 msgr2=0x7fa71806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 shutdown_connections 2024-09-12T02:18:16.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:16.435+0000 7fa71fda9640 1 -- 172.21.15.3:0/2143220795 wait complete. 2024-09-12T02:18:16.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:16 smithi026 ceph-mon[35496]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:16.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:16 smithi003 ceph-mon[31476]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:16.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:16 smithi092 ceph-mon[36089]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:17.170 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T02:18:17.179 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T02:18:17.179 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T02:18:17.179 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T02:18:17.179 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T02:18:17.206 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T02:18:17.207 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T02:18:17.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:17.207 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T02:18:17.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:17.225 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-12T02:18:17.294 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-12T02:18:17.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:18:17.297 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T02:18:17.297 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T02:18:17.297 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:17.297 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T02:18:17.592 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 -- 172.21.15.3:0/4208459142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 msgr2=0x7f6e1410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/4208459142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e1410bb50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6e08009900 tx=0x7f6e0802a480 comp rx=0 tx=0).stop 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 -- 172.21.15.3:0/4208459142 shutdown_connections 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/4208459142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 0x7f6e1410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/4208459142 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e1410bb50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/4208459142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 0x7f6e14103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 -- 172.21.15.3:0/4208459142 >> 172.21.15.3:0/4208459142 conn(0x7f6e140fc7a0 msgr2=0x7f6e140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:18.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 -- 172.21.15.3:0/4208459142 shutdown_connections 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.053+0000 7f6e1a71a640 1 -- 172.21.15.3:0/4208459142 wait complete. 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.054+0000 7f6e1a71a640 1 Processor -- start 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.054+0000 7f6e1a71a640 1 -- start start 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 0x7f6e14198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 0x7f6e1419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14111f70 con 0x7f6e14103660 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14111df0 con 0x7f6e14102d40 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e1a71a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141120f0 con 0x7f6e1410c170 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60568/0 (socket says 172.21.15.3:60568) 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 -- 172.21.15.3:0/1387931229 learned_addr learned my addr 172.21.15.3:0/1387931229 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:18.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e13fff640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 0x7f6e14198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e18cb7640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 0x7f6e1419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 -- 172.21.15.3:0/1387931229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 msgr2=0x7f6e1419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 0x7f6e1419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 -- 172.21.15.3:0/1387931229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 msgr2=0x7f6e14198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 0x7f6e14198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 -- 172.21.15.3:0/1387931229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1419d420 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e137fe640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6e08006010 tx=0x7f6e080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.055+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0803f030 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.056+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1419d6b0 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.056+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e08003ce0 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.056+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08035070 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.056+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a4ff0 con 0x7f6e14103660 2024-09-12T02:18:18.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.057+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1406b070 con 0x7f6e14103660 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.058+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6e08042030 con 0x7f6e14103660 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.058+0000 7f6e117fa640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 0x7f6dfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.059+0000 7f6e13fff640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 0x7f6dfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.059+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6e080ef750 con 0x7f6e14103660 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.059+0000 7f6e13fff640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 0x7f6dfc0a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400bbe0 tx=0x7f6e0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:18.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.062+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e080bc110 con 0x7f6e14103660 2024-09-12T02:18:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.277+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e14103120 con 0x7f6e14103660 2024-09-12T02:18:18.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.277+0000 7f6e117fa640 1 -- 172.21.15.3:0/1387931229 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f6e14103120 con 0x7f6e14103660 2024-09-12T02:18:18.278 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.279+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 msgr2=0x7f6dfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.279+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 0x7f6dfc0a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400bbe0 tx=0x7f6e0400b390 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 msgr2=0x7f6e141989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6e08006010 tx=0x7f6e080042f0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 shutdown_connections 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6dfc0a2b80 0x7f6dfc0a5040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e1410c170 0x7f6e1419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14103660 0x7f6e141989b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 --2- 172.21.15.3:0/1387931229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e14102d40 0x7f6e14198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 >> 172.21.15.3:0/1387931229 conn(0x7f6e140fc7a0 msgr2=0x7f6e140fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 shutdown_connections 2024-09-12T02:18:18.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:18.280+0000 7f6e1a71a640 1 -- 172.21.15.3:0/1387931229 wait complete. 2024-09-12T02:18:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:18 smithi003 ceph-mon[31476]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:18 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1387931229' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:18:18.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:18 smithi092 ceph-mon[36089]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:18.591 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:18 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1387931229' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:18:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:18 smithi026 ceph-mon[35496]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:18 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1387931229' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:18:19.070 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T02:18:19.071 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:18:19.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:19.071 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-09-12T02:18:19.091 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T02:18:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T02:18:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T02:18:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:18:19.091 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-09-12T02:18:19.092 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link 2024-09-12T02:18:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:18:19.093 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:18:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-09-12T02:18:19.093 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-09-12T02:18:19.093 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T02:18:19.093 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:18:19.093 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T02:18:19.093 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-09-12T02:18:19.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-09-12T02:18:19.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-09-12T02:18:19.094 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T02:18:19.094 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T02:18:19.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:19.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:19.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:19.309 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T02:18:19.336 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-09-12T02:18:19.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:19.338 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-09-12T02:18:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-09-12T02:18:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T02:18:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T02:18:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T02:18:19.418 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T02:18:19.419 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T02:18:19.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:19.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:19.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:19.713 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T02:18:19.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:19.730 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-09-12T02:18:19.786 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:18:19.786 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T02:18:19.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:18:19.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T02:18:19.786 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T02:18:19.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:19.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:19.906 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T02:18:19.906 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:18:19.906 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T02:18:19.907 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T02:18:19.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:20.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.053 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:18:20.053 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-09-12T02:18:20.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-09-12T02:18:20.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T02:18:20.053 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-09-12T02:18:20.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:20.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:18:20.134 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:20.150 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:18:20.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:18:20.150 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:20.207 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T02:18:20.207 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T02:18:20.208 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 2024-09-12T02:18:20.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:20.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T02:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T02:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T02:18:20.301 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-12T02:18:20.447 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T02:18:20.447 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:invalid new device string format 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:3300,172.21.15.26:3300,172.21.15.92:3300" 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.3:3300,172.21.15.26:3300,172.21.15.92:3300:/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=c595fa10-70ab-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T02:18:20.448 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-09-12T02:18:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T02:18:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T02:18:20.449 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T02:18:20.454 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T02:18:20.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:18:20.454 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:20.485 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:18:20.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:20 smithi003 ceph-mon[31476]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:20.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:18:20.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.584 INFO:teuthology.orchestra.run.smithi003.stdout:client24928 2024-09-12T02:18:20.584 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:18:20.584 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:18:20.603 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:18:20.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:18:20.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.703 INFO:teuthology.orchestra.run.smithi003.stdout:client24928 2024-09-12T02:18:20.703 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:18:20.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:18:20.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.763 INFO:teuthology.orchestra.run.smithi003.stdout:client24928 2024-09-12T02:18:20.764 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c595fa10-70ab-11ef-bcea-c7b262605968.client24928/status') 2024-09-12T02:18:20.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:20 smithi026 ceph-mon[35496]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:20 smithi092 ceph-mon[36089]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:20.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:18:20.824 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.24928 (3)172.21.15.3:0/499285940 2024-09-12T02:18:20.824 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-09-12T02:18:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-09-12T02:18:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-09-12T02:18:20.824 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 2.4112e-05 s, 2.9 MB/s 2024-09-12T02:18:20.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:18:20.831 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T02:18:20.841 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:18:20.842 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:18:20.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T02:18:20.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.947+0000 7f7bcb65e640 1 -- 172.21.15.3:0/3937265629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 msgr2=0x7f7bc4109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.947+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/3937265629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc4109f80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009ca0 tx=0x7f7bbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- 172.21.15.3:0/3937265629 shutdown_connections 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/3937265629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc4109f80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/3937265629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4102d30 0x7f7bc41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/3937265629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4101b40 0x7f7bc4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- 172.21.15.3:0/3937265629 >> 172.21.15.3:0/3937265629 conn(0x7f7bc40fd2e0 msgr2=0x7f7bc40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- 172.21.15.3:0/3937265629 shutdown_connections 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- 172.21.15.3:0/3937265629 wait complete. 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 Processor -- start 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- start start 2024-09-12T02:18:20.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4101b40 0x7f7bc4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102d30 0x7f7bc4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d7f0 con 0x7f7bc41036f0 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d670 con 0x7f7bc4101b40 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.948+0000 7f7bcb65e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d970 con 0x7f7bc4102d30 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58910/0 (socket says 172.21.15.3:58910) 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 -- 172.21.15.3:0/2705350868 learned_addr learned my addr 172.21.15.3:0/2705350868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc93fa640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4101b40 0x7f7bc4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc8bf9640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102d30 0x7f7bc4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 -- 172.21.15.3:0/2705350868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102d30 msgr2=0x7f7bc4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102d30 0x7f7bc4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 -- 172.21.15.3:0/2705350868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4101b40 msgr2=0x7f7bc4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:20.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4101b40 0x7f7bc4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 -- 172.21.15.3:0/2705350868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc419fa10 con 0x7f7bc41036f0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bc9bfb640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0319f0 tx=0x7f7bbc02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc003c70 con 0x7f7bc41036f0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc419fca0 con 0x7f7bc41036f0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bbc003e10 con 0x7f7bc41036f0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc02fd30 con 0x7f7bc41036f0 2024-09-12T02:18:20.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.949+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41a01a0 con 0x7f7bc41036f0 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.950+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc4108e60 con 0x7f7bc41036f0 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.951+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7bbc004040 con 0x7f7bc41036f0 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.952+0000 7f7bba7fc640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.952+0000 7f7bc93fa640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.952+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7bbc0f0710 con 0x7f7bc41036f0 2024-09-12T02:18:20.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.953+0000 7f7bc93fa640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 0x7f7bb00a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005d00 tx=0x7f7bc0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:20.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:20.956+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bbc0bd660 con 0x7f7bc41036f0 2024-09-12T02:18:21.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.172+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7bc4062e90 con 0x7f7bc41036f0 2024-09-12T02:18:21.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.337+0000 7f7bba7fc640 1 -- 172.21.15.3:0/2705350868 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f7bbc0bcf60 con 0x7f7bc41036f0 2024-09-12T02:18:21.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 msgr2=0x7f7bb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 0x7f7bb00a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005d00 tx=0x7f7bc0005c50 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 msgr2=0x7f7bc419f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0319f0 tx=0x7f7bbc02f450 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 shutdown_connections 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7bc41036f0 0x7f7bc419f3a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bc4102d30 0x7f7bc4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 --2- 172.21.15.3:0/2705350868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bc4101b40 0x7f7bc4197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 >> 172.21.15.3:0/2705350868 conn(0x7f7bc40fd2e0 msgr2=0x7f7bc4106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 shutdown_connections 2024-09-12T02:18:21.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.339+0000 7f7bcb65e640 1 -- 172.21.15.3:0/2705350868 wait complete. 2024-09-12T02:18:21.350 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T02:18:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 -- 172.21.15.3:0/2895104168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0104f20 msgr2=0x7f68a0107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 --2- 172.21.15.3:0/2895104168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0104f20 0x7f68a0107310 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f688c009900 tx=0x7f688c02a410 comp rx=0 tx=0).stop 2024-09-12T02:18:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 -- 172.21.15.3:0/2895104168 shutdown_connections 2024-09-12T02:18:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 --2- 172.21.15.3:0/2895104168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a0107850 0x7f68a0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 --2- 172.21.15.3:0/2895104168 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0104f20 0x7f68a0107310 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 --2- 172.21.15.3:0/2895104168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a01025f0 0x7f68a01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 -- 172.21.15.3:0/2895104168 >> 172.21.15.3:0/2895104168 conn(0x7f68a00fc1f0 msgr2=0x7f68a00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 -- 172.21.15.3:0/2895104168 shutdown_connections 2024-09-12T02:18:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.456+0000 7f68a71cf640 1 -- 172.21.15.3:0/2895104168 wait complete. 2024-09-12T02:18:21.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 Processor -- start 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 -- start start 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a01025f0 0x7f68a0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 0x7f68a0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68a010d850 con 0x7f68a0107850 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68a010d6d0 con 0x7f68a0104f20 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a71cf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68a010d9d0 con 0x7f68a01025f0 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58936/0 (socket says 172.21.15.3:58936) 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 -- 172.21.15.3:0/822502647 learned_addr learned my addr 172.21.15.3:0/822502647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:21.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a4f6b640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a01025f0 0x7f68a0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f689ffff640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 0x7f68a0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 -- 172.21.15.3:0/822502647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a01025f0 msgr2=0x7f68a0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a01025f0 0x7f68a0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 -- 172.21.15.3:0/822502647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 msgr2=0x7f68a0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 0x7f68a0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.457+0000 7f68a576c640 1 -- 172.21.15.3:0/822502647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a019fb50 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f689ffff640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 0x7f68a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f68a576c640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f689800dac0 tx=0x7f689800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898017070 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6898003e50 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a019fe40 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68980106c0 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.458+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a01a02b0 con 0x7f68a0107850 2024-09-12T02:18:21.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.459+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a00fdce0 con 0x7f68a0107850 2024-09-12T02:18:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.460+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f6898003970 con 0x7f68a0107850 2024-09-12T02:18:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.461+0000 7f689dffb640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 0x7f68900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.462+0000 7f68a4f6b640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 0x7f68900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.462+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f68980cea20 con 0x7f68a0107850 2024-09-12T02:18:21.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.462+0000 7f68a4f6b640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 0x7f68900a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6894009bd0 tx=0x7f6894005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.466+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f689809b910 con 0x7f68a0107850 2024-09-12T02:18:21.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.669+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f68a0062790 con 0x7f68900a2b80 2024-09-12T02:18:21.675 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:18:21.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.686+0000 7f689dffb640 1 -- 172.21.15.3:0/822502647 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f68a0062790 con 0x7f68900a2b80 2024-09-12T02:18:21.686 INFO:teuthology.orchestra.run.smithi003.stdout:{} 2024-09-12T02:18:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 msgr2=0x7f68900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 0x7f68900a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6894009bd0 tx=0x7f6894005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 msgr2=0x7f68a019f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f689800dac0 tx=0x7f689800df80 comp rx=0 tx=0).stop 2024-09-12T02:18:21.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 shutdown_connections 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68900a2b80 0x7f68900a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68a0107850 0x7f68a019f4e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68a0104f20 0x7f68a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 --2- 172.21.15.3:0/822502647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68a01025f0 0x7f68a0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 >> 172.21.15.3:0/822502647 conn(0x7f68a00fc1f0 msgr2=0x7f68a0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 shutdown_connections 2024-09-12T02:18:21.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.688+0000 7f68a71cf640 1 -- 172.21.15.3:0/822502647 wait complete. 2024-09-12T02:18:21.699 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T02:18:21.711 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T02:18:21.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:18:21.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:21 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:18:21.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:21 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:18:21.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- 172.21.15.3:0/1632478558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c102600 msgr2=0x7f066c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- 172.21.15.3:0/1632478558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c102600 0x7f066c108e90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0664009ca0 tx=0x7f066402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- 172.21.15.3:0/1632478558 shutdown_connections 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- 172.21.15.3:0/1632478558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c102600 0x7f066c108e90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- 172.21.15.3:0/1632478558 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c101c40 0x7f066c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- 172.21.15.3:0/1632478558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 0x7f066c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- 172.21.15.3:0/1632478558 >> 172.21.15.3:0/1632478558 conn(0x7f066c0fc1f0 msgr2=0x7f066c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- 172.21.15.3:0/1632478558 shutdown_connections 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- 172.21.15.3:0/1632478558 wait complete. 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 Processor -- start 2024-09-12T02:18:21.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 -- start start 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 0x7f066c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.825+0000 7f0673ad2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 0x7f066c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d820 con 0x7f066c101c40 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d6a0 con 0x7f066c100a50 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f066c10d9a0 con 0x7f066c102600 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58964/0 (socket says 172.21.15.3:58964) 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 -- 172.21.15.3:0/3183461327 learned_addr learned my addr 172.21.15.3:0/3183461327 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067186e640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 0x7f066c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067206f640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 0x7f066c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 -- 172.21.15.3:0/3183461327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 msgr2=0x7f066c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 0x7f066c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 -- 172.21.15.3:0/3183461327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 msgr2=0x7f066c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 0x7f066c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 -- 172.21.15.3:0/3183461327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c19f9f0 con 0x7f066c101c40 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067206f640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 0x7f066c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f067106d640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9f0 tx=0x7f065c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00cd80 con 0x7f066c101c40 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c19fce0 con 0x7f066c101c40 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c005500 con 0x7f066c101c40 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c010690 con 0x7f066c101c40 2024-09-12T02:18:21.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.826+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a0180 con 0x7f066c101c40 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.827+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c107d90 con 0x7f066c101c40 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.828+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f065c0046c0 con 0x7f066c101c40 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.829+0000 7f0662ffd640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 0x7f06580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.829+0000 7f067186e640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 0x7f06580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.829+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f065c0cf520 con 0x7f066c101c40 2024-09-12T02:18:21.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.830+0000 7f067186e640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 0x7f06580a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f066c101aa0 tx=0x7f0668005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:21.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:21.833+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f065c09c470 con 0x7f066c101c40 2024-09-12T02:18:22.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.010+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f066c062e90 con 0x7f066c101c40 2024-09-12T02:18:22.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.012+0000 7f0662ffd640 1 -- 172.21.15.3:0/3183461327 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f065c09bd70 con 0x7f066c101c40 2024-09-12T02:18:22.012 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:22.012 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 msgr2=0x7f06580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 0x7f06580a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f066c101aa0 tx=0x7f0668005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 msgr2=0x7f066c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e9f0 tx=0x7f065c00edf0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 shutdown_connections 2024-09-12T02:18:22.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f06580a2b90 0x7f06580a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f066c102600 0x7f066c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f066c101c40 0x7f066c198310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 --2- 172.21.15.3:0/3183461327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f066c100a50 0x7f066c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 >> 172.21.15.3:0/3183461327 conn(0x7f066c0fc1f0 msgr2=0x7f066c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 shutdown_connections 2024-09-12T02:18:22.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.014+0000 7f0673ad2640 1 -- 172.21.15.3:0/3183461327 wait complete. 2024-09-12T02:18:22.024 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:18:21.674+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:18:22.024 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:18:21.674+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:18:22.024 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:18:21.674+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:18:22.024 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:18:21.674+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:18:22.024 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:18:21 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:18:21.674+0000 7fb5e8891640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:18:22.026 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:11:52.858011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T02:16:25.722169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T02:16:28.093977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:18:22.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 -- 172.21.15.3:0/716099722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 msgr2=0x7ff4401020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 --2- 172.21.15.3:0/716099722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff4401020c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff430009900 tx=0x7ff43002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 -- 172.21.15.3:0/716099722 shutdown_connections 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 --2- 172.21.15.3:0/716099722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 0x7ff440108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 --2- 172.21.15.3:0/716099722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff4401020c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 --2- 172.21.15.3:0/716099722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 0x7ff440100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.135 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 -- 172.21.15.3:0/716099722 >> 172.21.15.3:0/716099722 conn(0x7ff4400fc190 msgr2=0x7ff4400fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 -- 172.21.15.3:0/716099722 shutdown_connections 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.133+0000 7ff44831f640 1 -- 172.21.15.3:0/716099722 wait complete. 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 Processor -- start 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 -- start start 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 0x7ff44010b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 0x7ff440109f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010d840 con 0x7ff440101c40 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010d6c0 con 0x7ff440100a50 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff44831f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff44010d9c0 con 0x7ff440102600 2024-09-12T02:18:22.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58972/0 (socket says 172.21.15.3:58972) 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 -- 172.21.15.3:0/1930089076 learned_addr learned my addr 172.21.15.3:0/1930089076 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4468bc640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 0x7ff440109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4460bb640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 0x7ff44010b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 -- 172.21.15.3:0/1930089076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 msgr2=0x7ff440109f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 0x7ff440109f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 -- 172.21.15.3:0/1930089076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 msgr2=0x7ff44010b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 0x7ff44010b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 -- 172.21.15.3:0/1930089076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44010a7e0 con 0x7ff440101c40 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.134+0000 7ff4458ba640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff430005940 tx=0x7ff430004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.135+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43003f030 con 0x7ff440101c40 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.135+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff430003ce0 con 0x7ff440101c40 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.135+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4401a1ff0 con 0x7ff440101c40 2024-09-12T02:18:22.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.135+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430035070 con 0x7ff440101c40 2024-09-12T02:18:22.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.135+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4401a24e0 con 0x7ff440101c40 2024-09-12T02:18:22.138 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.136+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff440107e90 con 0x7ff440101c40 2024-09-12T02:18:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.137+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff430042030 con 0x7ff440101c40 2024-09-12T02:18:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.138+0000 7ff4377fe640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 0x7ff42c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.138+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff4300ef850 con 0x7ff440101c40 2024-09-12T02:18:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.138+0000 7ff4460bb640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 0x7ff42c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.138+0000 7ff4460bb640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 0x7ff42c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff440101aa0 tx=0x7ff43c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.142+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4300bc150 con 0x7ff440101c40 2024-09-12T02:18:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.320+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff440062df0 con 0x7ff440101c40 2024-09-12T02:18:22.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.320+0000 7ff4377fe640 1 -- 172.21.15.3:0/1930089076 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7ff4300c0450 con 0x7ff440101c40 2024-09-12T02:18:22.320 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T02:18:22.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 msgr2=0x7ff42c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 0x7ff42c0a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff440101aa0 tx=0x7ff43c00b390 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 msgr2=0x7ff440109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff430005940 tx=0x7ff430004360 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 shutdown_connections 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff42c0a2c10 0x7ff42c0a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff440102600 0x7ff440109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff440101c40 0x7ff440109a10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 --2- 172.21.15.3:0/1930089076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440100a50 0x7ff44010b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 >> 172.21.15.3:0/1930089076 conn(0x7ff4400fc190 msgr2=0x7ff440105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.322+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 shutdown_connections 2024-09-12T02:18:22.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.323+0000 7ff44831f640 1 -- 172.21.15.3:0/1930089076 wait complete. 2024-09-12T02:18:22.333 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 -- 172.21.15.3:0/3640513081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 msgr2=0x7f70a81072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 --2- 172.21.15.3:0/3640513081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a81072d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7094009900 tx=0x7f709402a410 comp rx=0 tx=0).stop 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 -- 172.21.15.3:0/3640513081 shutdown_connections 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 --2- 172.21.15.3:0/3640513081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 --2- 172.21.15.3:0/3640513081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a81072d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 --2- 172.21.15.3:0/3640513081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 0x7f70a81049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 -- 172.21.15.3:0/3640513081 >> 172.21.15.3:0/3640513081 conn(0x7f70a80fc190 msgr2=0x7f70a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 -- 172.21.15.3:0/3640513081 shutdown_connections 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.440+0000 7f70aea82640 1 -- 172.21.15.3:0/3640513081 wait complete. 2024-09-12T02:18:22.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 Processor -- start 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 -- start start 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 0x7f70a8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810d7e0 con 0x7f70a8104ee0 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810d660 con 0x7f70a8107810 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70aea82640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810d960 con 0x7f70a81025b0 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59000/0 (socket says 172.21.15.3:59000) 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 -- 172.21.15.3:0/981344690 learned_addr learned my addr 172.21.15.3:0/981344690 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70ac81e640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 0x7f70a8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70ad01f640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 -- 172.21.15.3:0/981344690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 msgr2=0x7f70a8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 0x7f70a8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 -- 172.21.15.3:0/981344690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 msgr2=0x7f70a819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70a7fff640 1 -- 172.21.15.3:0/981344690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a819fb10 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.441+0000 7f70ad01f640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70a7fff640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7094031610 tx=0x7f70940043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709403f030 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7094003ce0 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a819fda0 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094035070 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.442+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a81a0230 con 0x7f70a8104ee0 2024-09-12T02:18:22.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.443+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a80fdc80 con 0x7f70a8104ee0 2024-09-12T02:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.444+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7094042030 con 0x7f70a8104ee0 2024-09-12T02:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.445+0000 7f70a5ffb640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 0x7f70980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.445+0000 7f70ac81e640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 0x7f70980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.445+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f70940ef720 con 0x7f70a8104ee0 2024-09-12T02:18:22.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.446+0000 7f70ac81e640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 0x7f70980a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f709c005fd0 tx=0x7f709c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.449+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70940bc610 con 0x7f70a8104ee0 2024-09-12T02:18:22.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.629+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70a8062e90 con 0x7f70a8104ee0 2024-09-12T02:18:22.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.629+0000 7f70a5ffb640 1 -- 172.21.15.3:0/981344690 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f70940bbf10 con 0x7f70a8104ee0 2024-09-12T02:18:22.629 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-12T02:18:22.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 msgr2=0x7f70980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 0x7f70980a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f709c005fd0 tx=0x7f709c005850 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 msgr2=0x7f70a8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7094031610 tx=0x7f70940043b0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 shutdown_connections 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f70980a2b90 0x7f70980a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8107810 0x7f70a819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f70a8104ee0 0x7f70a8198430 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 --2- 172.21.15.3:0/981344690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70a81025b0 0x7f70a8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 >> 172.21.15.3:0/981344690 conn(0x7f70a80fc190 msgr2=0x7f70a8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 shutdown_connections 2024-09-12T02:18:22.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.631+0000 7f70aea82640 1 -- 172.21.15.3:0/981344690 wait complete. 2024-09-12T02:18:22.641 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:22 smithi003 ceph-mon[31476]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:22.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:22 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:18:22.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:22 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:18:22.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:22 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3183461327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:22 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1930089076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:18:22.643 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.749+0000 7f042a523640 1 -- 172.21.15.3:0/2988732648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424104f00 msgr2=0x7f04241072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.749+0000 7f042a523640 1 --2- 172.21.15.3:0/2988732648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424104f00 0x7f04241072f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0418009900 tx=0x7f041802a410 comp rx=0 tx=0).stop 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 -- 172.21.15.3:0/2988732648 shutdown_connections 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 --2- 172.21.15.3:0/2988732648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424107830 0x7f0424109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 --2- 172.21.15.3:0/2988732648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424104f00 0x7f04241072f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 --2- 172.21.15.3:0/2988732648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 0x7f04241049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 -- 172.21.15.3:0/2988732648 >> 172.21.15.3:0/2988732648 conn(0x7f04240fc190 msgr2=0x7f04240fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 -- 172.21.15.3:0/2988732648 shutdown_connections 2024-09-12T02:18:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 -- 172.21.15.3:0/2988732648 wait complete. 2024-09-12T02:18:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 Processor -- start 2024-09-12T02:18:22.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.750+0000 7f042a523640 1 -- start start 2024-09-12T02:18:22.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 0x7f0424197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424104f00 0x7f0424198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f042410d800 con 0x7f0424107830 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f042410d680 con 0x7f0424104f00 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f042410d980 con 0x7f04241025d0 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59020/0 (socket says 172.21.15.3:59020) 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 -- 172.21.15.3:0/338481169 learned_addr learned my addr 172.21.15.3:0/338481169 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0423fff640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 0x7f0424197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f04237fe640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424104f00 0x7f0424198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 -- 172.21.15.3:0/338481169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 msgr2=0x7f0424197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 0x7f0424197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 -- 172.21.15.3:0/338481169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424104f00 msgr2=0x7f0424198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424104f00 0x7f0424198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 -- 172.21.15.3:0/338481169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042419fb00 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f0428ac0640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f041c00ccb0 tx=0x7f041c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f450 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041c00fad0 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042419fdf0 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.751+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c018bf0 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.752+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04241a0260 con 0x7f0424107830 2024-09-12T02:18:22.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.752+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04240fdc80 con 0x7f0424107830 2024-09-12T02:18:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.754+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f041c00f5f0 con 0x7f0424107830 2024-09-12T02:18:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.754+0000 7f04217fa640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.755+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f041c0cf700 con 0x7f0424107830 2024-09-12T02:18:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.755+0000 7f0423fff640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:22.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.755+0000 7f0423fff640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 0x7f040c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0414005fd0 tx=0x7f041400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:22.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.757+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f041c014030 con 0x7f0424107830 2024-09-12T02:18:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:22 smithi026 ceph-mon[35496]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:22 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:18:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:22 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:18:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:22 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3183461327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:22 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1930089076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:18:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:22 smithi092 ceph-mon[36089]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-12T02:18:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:22 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2705350868' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:18:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:22 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:18:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:22 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3183461327' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:22 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1930089076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:18:22.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.933+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f042402da10 con 0x7f0424107830 2024-09-12T02:18:22.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.934+0000 7f04217fa640 1 -- 172.21.15.3:0/338481169 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f041c09c6e0 con 0x7f0424107830 2024-09-12T02:18:22.934 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 msgr2=0x7f040c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 0x7f040c0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0414005fd0 tx=0x7f041400b390 comp rx=0 tx=0).stop 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 msgr2=0x7f042419f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f041c00ccb0 tx=0x7f041c00a620 comp rx=0 tx=0).stop 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 shutdown_connections 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f040c0a2b90 0x7f040c0a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0424107830 0x7f042419f490 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0424104f00 0x7f0424198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 --2- 172.21.15.3:0/338481169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04241025d0 0x7f0424197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:22.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 >> 172.21.15.3:0/338481169 conn(0x7f04240fc190 msgr2=0x7f0424102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:22.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 shutdown_connections 2024-09-12T02:18:22.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:22.936+0000 7f042a523640 1 -- 172.21.15.3:0/338481169 wait complete. 2024-09-12T02:18:22.947 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T02:18:22.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:23.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:23 smithi003 ceph-mon[31476]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:23.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:23 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/981344690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.741 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:23 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/338481169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:23 smithi026 ceph-mon[35496]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:23.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:23 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/981344690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:23 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/338481169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:23 smithi092 ceph-mon[36089]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:18:23.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:23 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/981344690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:23 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/338481169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.824+0000 7f7b11a54640 1 -- 172.21.15.3:0/2344962159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c102790 msgr2=0x7f7b0c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.824+0000 7f7b11a54640 1 --2- 172.21.15.3:0/2344962159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c102790 0x7f7b0c1067e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 -- 172.21.15.3:0/2344962159 shutdown_connections 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 --2- 172.21.15.3:0/2344962159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 --2- 172.21.15.3:0/2344962159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c102790 0x7f7b0c1067e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 --2- 172.21.15.3:0/2344962159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c101de0 0x7f7b0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 -- 172.21.15.3:0/2344962159 >> 172.21.15.3:0/2344962159 conn(0x7f7b0c0fd8c0 msgr2=0x7f7b0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 -- 172.21.15.3:0/2344962159 shutdown_connections 2024-09-12T02:18:23.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 -- 172.21.15.3:0/2344962159 wait complete. 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 Processor -- start 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.825+0000 7f7b11a54640 1 -- start start 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 0x7f7b0c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c111f70 con 0x7f7b0c101de0 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c111df0 con 0x7f7b0c106f10 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c1120f0 con 0x7f7b0c102790 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59044/0 (socket says 172.21.15.3:59044) 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 -- 172.21.15.3:0/327971874 learned_addr learned my addr 172.21.15.3:0/327971874 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0b7fe640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:23.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0a7fc640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 0x7f7b0c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 -- 172.21.15.3:0/327971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 msgr2=0x7f7b0c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 0x7f7b0c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 -- 172.21.15.3:0/327971874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 msgr2=0x7f7b0c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 -- 172.21.15.3:0/327971874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c19d400 con 0x7f7b0c101de0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0b7fe640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0a7fc640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 0x7f7b0c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b0affd640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d950 tx=0x7f7afc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017070 con 0x7f7b0c101de0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c19d6f0 con 0x7f7b0c101de0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7afc009c30 con 0x7f7b0c101de0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00f6b0 con 0x7f7b0c101de0 2024-09-12T02:18:23.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.826+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c1a4fd0 con 0x7f7b0c101de0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.828+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c1021c0 con 0x7f7b0c101de0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.829+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f7afc0109d0 con 0x7f7b0c101de0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.829+0000 7f7b10a52640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 0x7f7af40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.829+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7afc0cef00 con 0x7f7b0c101de0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.829+0000 7f7b0a7fc640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 0x7f7af40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.830+0000 7f7b0a7fc640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 0x7f7af40a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b00032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:23.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:23.834+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7afc009de0 con 0x7f7b0c101de0 2024-09-12T02:18:24.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.046+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b0c10a1b0 con 0x7f7b0c101de0 2024-09-12T02:18:24.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.047+0000 7f7b10a52640 1 -- 172.21.15.3:0/327971874 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f7afc09bcf0 con 0x7f7b0c101de0 2024-09-12T02:18:24.047 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:24.047 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:18:24.048 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:18:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 msgr2=0x7f7af40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 0x7f7af40a50d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b00032c70 comp rx=0 tx=0).stop 2024-09-12T02:18:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 msgr2=0x7f7b0c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d950 tx=0x7f7afc00de10 comp rx=0 tx=0).stop 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 shutdown_connections 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7af40a2c10 0x7f7af40a50d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b0c106f10 0x7f7b0c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b0c102790 0x7f7b0c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 --2- 172.21.15.3:0/327971874 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b0c101de0 0x7f7b0c198450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 >> 172.21.15.3:0/327971874 conn(0x7f7b0c0fd8c0 msgr2=0x7f7b0c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 shutdown_connections 2024-09-12T02:18:24.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.049+0000 7f7b11a54640 1 -- 172.21.15.3:0/327971874 wait complete. 2024-09-12T02:18:24.705 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:24 smithi003 ceph-mon[31476]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:24.705 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:24 smithi003 ceph-mon[31476]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:18:24.705 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:24 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/327971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:24.736 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T02:18:24.736 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T02:18:24.736 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:18:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:24 smithi026 ceph-mon[35496]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:24 smithi026 ceph-mon[35496]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:18:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:24 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/327971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:24.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:24 smithi092 ceph-mon[36089]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:24.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:24 smithi092 ceph-mon[36089]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:18:24.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:24 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/327971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 -- 172.21.15.3:0/1694560321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c101b40 msgr2=0x7f136c101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 --2- 172.21.15.3:0/1694560321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c101b40 0x7f136c101f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1368009900 tx=0x7f136802a410 comp rx=0 tx=0).stop 2024-09-12T02:18:24.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 -- 172.21.15.3:0/1694560321 shutdown_connections 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 --2- 172.21.15.3:0/1694560321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c1036f0 0x7f136c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 --2- 172.21.15.3:0/1694560321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c102d30 0x7f136c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 --2- 172.21.15.3:0/1694560321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c101b40 0x7f136c101f40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 -- 172.21.15.3:0/1694560321 >> 172.21.15.3:0/1694560321 conn(0x7f136c0fd2a0 msgr2=0x7f136c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 -- 172.21.15.3:0/1694560321 shutdown_connections 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 -- 172.21.15.3:0/1694560321 wait complete. 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.845+0000 7f1373ce4640 1 Processor -- start 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 -- start start 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 0x7f136c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 0x7f136c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d7f0 con 0x7f136c102d30 2024-09-12T02:18:24.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d670 con 0x7f136c1036f0 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f136c10d970 con 0x7f136c101b40 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59064/0 (socket says 172.21.15.3:59064) 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 -- 172.21.15.3:0/3328808646 learned_addr learned my addr 172.21.15.3:0/3328808646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1371a80640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 0x7f136c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1372281640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 0x7f136c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 -- 172.21.15.3:0/3328808646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 msgr2=0x7f136c197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 0x7f136c197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 -- 172.21.15.3:0/3328808646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 msgr2=0x7f136c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 0x7f136c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 -- 172.21.15.3:0/3328808646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136c19fa00 con 0x7f136c102d30 2024-09-12T02:18:24.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1371a80640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 0x7f136c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1372281640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 0x7f136c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f137127f640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9f0 tx=0x7f135c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c00cd80 con 0x7f136c102d30 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135c010910 con 0x7f136c102d30 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c018990 con 0x7f136c102d30 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.846+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136c19fcf0 con 0x7f136c102d30 2024-09-12T02:18:24.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.847+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136c1a0230 con 0x7f136c102d30 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.848+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136c108ee0 con 0x7f136c102d30 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.849+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f135c010ac0 con 0x7f136c102d30 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.849+0000 7f1362ffd640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 0x7f13580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.850+0000 7f1371a80640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 0x7f13580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.850+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f135c0cf5a0 con 0x7f136c102d30 2024-09-12T02:18:24.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.850+0000 7f1371a80640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 0x7f13580a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f136802aa00 tx=0x7f1368000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:24.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:24.854+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f135c09c390 con 0x7f136c102d30 2024-09-12T02:18:25.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.029+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f136c062630 con 0x7f13580a2ca0 2024-09-12T02:18:25.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.036+0000 7f1362ffd640 1 -- 172.21.15.3:0/3328808646 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259109 (secure 0 0 0) 0x7f136c062630 con 0x7f13580a2ca0 2024-09-12T02:18:25.039 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:25.040 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 msgr2=0x7f13580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 0x7f13580a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f136802aa00 tx=0x7f1368000bc0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 msgr2=0x7f136c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9f0 tx=0x7f135c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 shutdown_connections 2024-09-12T02:18:25.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13580a2ca0 0x7f13580a5160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f136c1036f0 0x7f136c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f136c102d30 0x7f136c198320 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 --2- 172.21.15.3:0/3328808646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f136c101b40 0x7f136c197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 >> 172.21.15.3:0/3328808646 conn(0x7f136c0fd2a0 msgr2=0x7f136c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 shutdown_connections 2024-09-12T02:18:25.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.039+0000 7f1373ce4640 1 -- 172.21.15.3:0/3328808646 wait complete. 2024-09-12T02:18:25.050 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-12T02:18:24.457626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660260,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1184,"num_read_kb":712,"num_write":440,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360440,"kb_used_data":11192,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581384,"statfs":{"total":1151940427776,"available":1151571337216,"internally_reserved":0,"allocated":11460608,"data_stored":7237377,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":701,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451322+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451322+0000","last_peered":"2024-09-12T02:18:06.451322+0000","last_clean":"2024-09-12T02:18:06.451322+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451322+0000","last_undegraded":"2024-09-12T02:18:06.451322+0000","last_fullsized":"2024-09-12T02:18:06.451322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:20.881605+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:20.881605+0000","last_peered":"2024-09-12T02:18:20.881605+0000","last_clean":"2024-09-12T02:18:20.881605+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:20.881605+0000","last_undegraded":"2024-09-12T02:18:20.881605+0000","last_fullsized":"2024-09-12T02:18:20.881605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":226,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877832+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:18.877832+0000","last_peered":"2024-09-12T02:18:18.877832+0000","last_clean":"2024-09-12T02:18:18.877832+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:18.877832+0000","last_undegraded":"2024-09-12T02:18:18.877832+0000","last_fullsized":"2024-09-12T02:18:18.877832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":266,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877824+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:18.877824+0000","last_peered":"2024-09-12T02:18:18.877824+0000","last_clean":"2024-09-12T02:18:18.877824+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:18.877824+0000","last_undegraded":"2024-09-12T02:18:18.877824+0000","last_fullsized":"2024-09-12T02:18:18.877824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62001,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":165,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.883192+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:21.883192+0000","last_peered":"2024-09-12T02:18:21.883192+0000","last_clean":"2024-09-12T02:18:21.883192+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:21.883192+0000","last_undegraded":"2024-09-12T02:18:21.883192+0000","last_fullsized":"2024-09-12T02:18:21.883192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:19.880042+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:19.880042+0000","last_peered":"2024-09-12T02:18:19.880042+0000","last_clean":"2024-09-12T02:18:19.880042+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:19.880042+0000","last_undegraded":"2024-09-12T02:18:19.880042+0000","last_fullsized":"2024-09-12T02:18:19.880042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.882643+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:21.882643+0000","last_peered":"2024-09-12T02:18:21.882643+0000","last_clean":"2024-09-12T02:18:21.882643+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:21.882643+0000","last_undegraded":"2024-09-12T02:18:21.882643+0000","last_fullsized":"2024-09-12T02:18:21.882643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447475+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447475+0000","last_peered":"2024-09-12T02:18:06.447475+0000","last_clean":"2024-09-12T02:18:06.447475+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447475+0000","last_undegraded":"2024-09-12T02:18:06.447475+0000","last_fullsized":"2024-09-12T02:18:06.447475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.881952+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:21.881952+0000","last_peered":"2024-09-12T02:18:21.881952+0000","last_clean":"2024-09-12T02:18:21.881952+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:21.881952+0000","last_undegraded":"2024-09-12T02:18:21.881952+0000","last_fullsized":"2024-09-12T02:18:21.881952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'15","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:56.125345+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:56.125345+0000","last_peered":"2024-09-12T02:16:56.125345+0000","last_clean":"2024-09-12T02:16:56.125345+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:56.125345+0000","last_undegraded":"2024-09-12T02:16:56.125345+0000","last_fullsized":"2024-09-12T02:16:56.125345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460626+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.460626+0000","last_peered":"2024-09-12T02:18:06.460626+0000","last_clean":"2024-09-12T02:18:06.460626+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.460626+0000","last_undegraded":"2024-09-12T02:18:06.460626+0000","last_fullsized":"2024-09-12T02:18:06.460626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877138+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:18.877138+0000","last_peered":"2024-09-12T02:18:18.877138+0000","last_clean":"2024-09-12T02:18:18.877138+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:18.877138+0000","last_undegraded":"2024-09-12T02:18:18.877138+0000","last_fullsized":"2024-09-12T02:18:18.877138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69658,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":603,"num_write":257,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1091945,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":501696,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":506632,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:25.056 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 -- 172.21.15.3:0/3719355830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0101b10 msgr2=0x7f38c0101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 --2- 172.21.15.3:0/3719355830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0101b10 0x7f38c0101f90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 -- 172.21.15.3:0/3719355830 shutdown_connections 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 --2- 172.21.15.3:0/3719355830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c01024d0 0x7f38c0108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 --2- 172.21.15.3:0/3719355830 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0101b10 0x7f38c0101f90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 --2- 172.21.15.3:0/3719355830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c0100920 0x7f38c0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 -- 172.21.15.3:0/3719355830 >> 172.21.15.3:0/3719355830 conn(0x7f38c00fc0c0 msgr2=0x7f38c00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.194+0000 7f38c759d640 1 -- 172.21.15.3:0/3719355830 shutdown_connections 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 -- 172.21.15.3:0/3719355830 wait complete. 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 Processor -- start 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 -- start start 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 0x7f38c019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 0x7f38c01a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c5a0 con 0x7f38c0100920 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c420 con 0x7f38c0101b10 2024-09-12T02:18:25.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c759d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010c720 con 0x7f38c01024d0 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c659b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c659b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59080/0 (socket says 172.21.15.3:59080) 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.195+0000 7f38c659b640 1 -- 172.21.15.3:0/516003070 learned_addr learned my addr 172.21.15.3:0/516003070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c6d9c640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 0x7f38c01a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 -- 172.21.15.3:0/516003070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 msgr2=0x7f38c01a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 0x7f38c01a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c5d9a640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 0x7f38c019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 -- 172.21.15.3:0/516003070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 msgr2=0x7f38c019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 0x7f38c019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 -- 172.21.15.3:0/516003070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c01a1b00 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c6d9c640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 0x7f38c01a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c5d9a640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 0x7f38c019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c659b640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c970 tx=0x7f38bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004220 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc0043c0 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c01a1df0 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc004a00 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.196+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c01a2200 con 0x7f38c0100920 2024-09-12T02:18:25.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.197+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c01a24c0 con 0x7f38c0100920 2024-09-12T02:18:25.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.198+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f38bc0079b0 con 0x7f38c0100920 2024-09-12T02:18:25.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.199+0000 7f38b77fe640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 0x7f38ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.199+0000 7f38c5d9a640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 0x7f38ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.199+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f38bc0cf4a0 con 0x7f38c0100920 2024-09-12T02:18:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.200+0000 7f38c5d9a640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 0x7f38ac0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:25.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.202+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38bc014030 con 0x7f38c0100920 2024-09-12T02:18:25.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.375+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38c01a2690 con 0x7f38ac0a2c10 2024-09-12T02:18:25.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.381+0000 7f38b77fe640 1 -- 172.21.15.3:0/516003070 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259109 (secure 0 0 0) 0x7f38c01a2690 con 0x7f38ac0a2c10 2024-09-12T02:18:25.384 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:25.385 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 msgr2=0x7f38ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 0x7f38ac0a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0031400 comp rx=0 tx=0).stop 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 msgr2=0x7f38c0199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c970 tx=0x7f38bc00ce30 comp rx=0 tx=0).stop 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 shutdown_connections 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f38ac0a2c10 0x7f38ac0a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38c01024d0 0x7f38c01a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38c0101b10 0x7f38c019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 --2- 172.21.15.3:0/516003070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f38c0100920 0x7f38c0199ee0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 >> 172.21.15.3:0/516003070 conn(0x7f38c00fc0c0 msgr2=0x7f38c00fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 shutdown_connections 2024-09-12T02:18:25.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.384+0000 7f38c759d640 1 -- 172.21.15.3:0/516003070 wait complete. 2024-09-12T02:18:25.395 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-12T02:18:24.457626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660260,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1184,"num_read_kb":712,"num_write":440,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360440,"kb_used_data":11192,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581384,"statfs":{"total":1151940427776,"available":1151571337216,"internally_reserved":0,"allocated":11460608,"data_stored":7237377,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":701,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451322+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451322+0000","last_peered":"2024-09-12T02:18:06.451322+0000","last_clean":"2024-09-12T02:18:06.451322+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451322+0000","last_undegraded":"2024-09-12T02:18:06.451322+0000","last_fullsized":"2024-09-12T02:18:06.451322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:20.881605+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:20.881605+0000","last_peered":"2024-09-12T02:18:20.881605+0000","last_clean":"2024-09-12T02:18:20.881605+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:20.881605+0000","last_undegraded":"2024-09-12T02:18:20.881605+0000","last_fullsized":"2024-09-12T02:18:20.881605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":226,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877832+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:18.877832+0000","last_peered":"2024-09-12T02:18:18.877832+0000","last_clean":"2024-09-12T02:18:18.877832+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:18.877832+0000","last_undegraded":"2024-09-12T02:18:18.877832+0000","last_fullsized":"2024-09-12T02:18:18.877832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":266,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877824+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:18.877824+0000","last_peered":"2024-09-12T02:18:18.877824+0000","last_clean":"2024-09-12T02:18:18.877824+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:18.877824+0000","last_undegraded":"2024-09-12T02:18:18.877824+0000","last_fullsized":"2024-09-12T02:18:18.877824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62001,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":165,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.883192+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:21.883192+0000","last_peered":"2024-09-12T02:18:21.883192+0000","last_clean":"2024-09-12T02:18:21.883192+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:21.883192+0000","last_undegraded":"2024-09-12T02:18:21.883192+0000","last_fullsized":"2024-09-12T02:18:21.883192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:19.880042+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:19.880042+0000","last_peered":"2024-09-12T02:18:19.880042+0000","last_clean":"2024-09-12T02:18:19.880042+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:19.880042+0000","last_undegraded":"2024-09-12T02:18:19.880042+0000","last_fullsized":"2024-09-12T02:18:19.880042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.882643+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:21.882643+0000","last_peered":"2024-09-12T02:18:21.882643+0000","last_clean":"2024-09-12T02:18:21.882643+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:21.882643+0000","last_undegraded":"2024-09-12T02:18:21.882643+0000","last_fullsized":"2024-09-12T02:18:21.882643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447475+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447475+0000","last_peered":"2024-09-12T02:18:06.447475+0000","last_clean":"2024-09-12T02:18:06.447475+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447475+0000","last_undegraded":"2024-09-12T02:18:06.447475+0000","last_fullsized":"2024-09-12T02:18:06.447475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.881952+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:21.881952+0000","last_peered":"2024-09-12T02:18:21.881952+0000","last_clean":"2024-09-12T02:18:21.881952+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:21.881952+0000","last_undegraded":"2024-09-12T02:18:21.881952+0000","last_fullsized":"2024-09-12T02:18:21.881952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'15","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:56.125345+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:56.125345+0000","last_peered":"2024-09-12T02:16:56.125345+0000","last_clean":"2024-09-12T02:16:56.125345+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:56.125345+0000","last_undegraded":"2024-09-12T02:16:56.125345+0000","last_fullsized":"2024-09-12T02:16:56.125345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460626+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.460626+0000","last_peered":"2024-09-12T02:18:06.460626+0000","last_clean":"2024-09-12T02:18:06.460626+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.460626+0000","last_undegraded":"2024-09-12T02:18:06.460626+0000","last_fullsized":"2024-09-12T02:18:06.460626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877138+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:18.877138+0000","last_peered":"2024-09-12T02:18:18.877138+0000","last_clean":"2024-09-12T02:18:18.877138+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:18.877138+0000","last_undegraded":"2024-09-12T02:18:18.877138+0000","last_fullsized":"2024-09-12T02:18:18.877138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69658,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":603,"num_write":257,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1091945,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":501696,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":506632,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:25.399 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T02:18:25.400 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:18:25.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- 172.21.15.3:0/4209449192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c074c50 msgr2=0x7f4f7c0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 --2- 172.21.15.3:0/4209449192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c074c50 0x7f4f7c0750b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009900 tx=0x7f4f6402a410 comp rx=0 tx=0).stop 2024-09-12T02:18:25.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- 172.21.15.3:0/4209449192 shutdown_connections 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 --2- 172.21.15.3:0/4209449192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c1099b0 0x7f4f7c10be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 --2- 172.21.15.3:0/4209449192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c074c50 0x7f4f7c0750b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 --2- 172.21.15.3:0/4209449192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c076230 0x7f4f7c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- 172.21.15.3:0/4209449192 >> 172.21.15.3:0/4209449192 conn(0x7f4f7c0fd2a0 msgr2=0x7f4f7c0ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- 172.21.15.3:0/4209449192 shutdown_connections 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- 172.21.15.3:0/4209449192 wait complete. 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 Processor -- start 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.541+0000 7f4f8205c640 1 -- start start 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c074c50 0x7f4f7c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c1099b0 0x7f4f7c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10f9b0 con 0x7f4f7c076230 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10f830 con 0x7f4f7c1099b0 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10fb30 con 0x7f4f7c074c50 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59102/0 (socket says 172.21.15.3:59102) 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 -- 172.21.15.3:0/1407720733 learned_addr learned my addr 172.21.15.3:0/1407720733 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f7b7fe640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c074c50 0x7f4f7c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f7bfff640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c1099b0 0x7f4f7c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 -- 172.21.15.3:0/1407720733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c074c50 msgr2=0x7f4f7c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c074c50 0x7f4f7c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 -- 172.21.15.3:0/1407720733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c1099b0 msgr2=0x7f4f7c1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c1099b0 0x7f4f7c1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 -- 172.21.15.3:0/1407720733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c1a1c50 con 0x7f4f7c076230 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f73fff640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009230 tx=0x7f4f640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6403f030 con 0x7f4f7c076230 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f64003ce0 con 0x7f4f7c076230 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.542+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c1a1ee0 con 0x7f4f7c076230 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.543+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64035070 con 0x7f4f7c076230 2024-09-12T02:18:25.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.543+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c1a2340 con 0x7f4f7c076230 2024-09-12T02:18:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.544+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c108730 con 0x7f4f7c076230 2024-09-12T02:18:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.545+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f4f64042030 con 0x7f4f7c076230 2024-09-12T02:18:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.546+0000 7f4f797fa640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 0x7f4f680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.546+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4f640ef600 con 0x7f4f7c076230 2024-09-12T02:18:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.546+0000 7f4f7b7fe640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 0x7f4f680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.546+0000 7f4f7b7fe640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 0x7f4f680a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009bd0 tx=0x7f4f6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.550+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f640bbda0 con 0x7f4f7c076230 2024-09-12T02:18:25.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.725+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f7c062630 con 0x7f4f680a2c20 2024-09-12T02:18:25.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.731+0000 7f4f797fa640 1 -- 172.21.15.3:0/1407720733 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259109 (secure 0 0 0) 0x7f4f7c062630 con 0x7f4f680a2c20 2024-09-12T02:18:25.733 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:25.735 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:25.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 msgr2=0x7f4f680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 0x7f4f680a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009bd0 tx=0x7f4f6c005f90 comp rx=0 tx=0).stop 2024-09-12T02:18:25.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 msgr2=0x7f4f7c19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:25.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009230 tx=0x7f4f640042f0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 shutdown_connections 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4f680a2c20 0x7f4f680a50e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f7c1099b0 0x7f4f7c1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4f7c076230 0x7f4f7c19a570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 --2- 172.21.15.3:0/1407720733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f7c074c50 0x7f4f7c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 >> 172.21.15.3:0/1407720733 conn(0x7f4f7c0fd2a0 msgr2=0x7f4f7c105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 shutdown_connections 2024-09-12T02:18:25.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:25.734+0000 7f4f8205c640 1 -- 172.21.15.3:0/1407720733 wait complete. 2024-09-12T02:18:25.745 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-12T02:18:24.457626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660260,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1184,"num_read_kb":712,"num_write":440,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":859,"ondisk_log_size":859,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360440,"kb_used_data":11192,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581384,"statfs":{"total":1151940427776,"available":1151571337216,"internally_reserved":0,"allocated":11460608,"data_stored":7237377,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":701,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451322+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451322+0000","last_peered":"2024-09-12T02:18:06.451322+0000","last_clean":"2024-09-12T02:18:06.451322+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451322+0000","last_undegraded":"2024-09-12T02:18:06.451322+0000","last_fullsized":"2024-09-12T02:18:06.451322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:20.881605+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:20.881605+0000","last_peered":"2024-09-12T02:18:20.881605+0000","last_clean":"2024-09-12T02:18:20.881605+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:20.881605+0000","last_undegraded":"2024-09-12T02:18:20.881605+0000","last_fullsized":"2024-09-12T02:18:20.881605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":226,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877832+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:18.877832+0000","last_peered":"2024-09-12T02:18:18.877832+0000","last_clean":"2024-09-12T02:18:18.877832+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:18.877832+0000","last_undegraded":"2024-09-12T02:18:18.877832+0000","last_fullsized":"2024-09-12T02:18:18.877832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":266,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877824+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:18.877824+0000","last_peered":"2024-09-12T02:18:18.877824+0000","last_clean":"2024-09-12T02:18:18.877824+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:18.877824+0000","last_undegraded":"2024-09-12T02:18:18.877824+0000","last_fullsized":"2024-09-12T02:18:18.877824+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62001,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":165,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.883192+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:21.883192+0000","last_peered":"2024-09-12T02:18:21.883192+0000","last_clean":"2024-09-12T02:18:21.883192+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:21.883192+0000","last_undegraded":"2024-09-12T02:18:21.883192+0000","last_fullsized":"2024-09-12T02:18:21.883192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":131,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:19.880042+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:19.880042+0000","last_peered":"2024-09-12T02:18:19.880042+0000","last_clean":"2024-09-12T02:18:19.880042+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:19.880042+0000","last_undegraded":"2024-09-12T02:18:19.880042+0000","last_fullsized":"2024-09-12T02:18:19.880042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.158489+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:38.158489+0000","last_peered":"2024-09-12T02:16:38.158489+0000","last_clean":"2024-09-12T02:16:38.158489+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:38.158489+0000","last_undegraded":"2024-09-12T02:16:38.158489+0000","last_fullsized":"2024-09-12T02:16:38.158489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.882643+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:21.882643+0000","last_peered":"2024-09-12T02:18:21.882643+0000","last_clean":"2024-09-12T02:18:21.882643+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:21.882643+0000","last_undegraded":"2024-09-12T02:18:21.882643+0000","last_fullsized":"2024-09-12T02:18:21.882643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447475+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447475+0000","last_peered":"2024-09-12T02:18:06.447475+0000","last_clean":"2024-09-12T02:18:06.447475+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447475+0000","last_undegraded":"2024-09-12T02:18:06.447475+0000","last_fullsized":"2024-09-12T02:18:06.447475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.881952+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:21.881952+0000","last_peered":"2024-09-12T02:18:21.881952+0000","last_clean":"2024-09-12T02:18:21.881952+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:21.881952+0000","last_undegraded":"2024-09-12T02:18:21.881952+0000","last_fullsized":"2024-09-12T02:18:21.881952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'15","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:56.125345+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:16:56.125345+0000","last_peered":"2024-09-12T02:16:56.125345+0000","last_clean":"2024-09-12T02:16:56.125345+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:16:56.125345+0000","last_undegraded":"2024-09-12T02:16:56.125345+0000","last_fullsized":"2024-09-12T02:16:56.125345+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.138455+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.138455+0000","last_peered":"2024-09-12T02:16:38.138455+0000","last_clean":"2024-09-12T02:16:38.138455+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.138455+0000","last_undegraded":"2024-09-12T02:16:38.138455+0000","last_fullsized":"2024-09-12T02:16:38.138455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460626+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.460626+0000","last_peered":"2024-09-12T02:18:06.460626+0000","last_clean":"2024-09-12T02:18:06.460626+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.460626+0000","last_undegraded":"2024-09-12T02:18:06.460626+0000","last_fullsized":"2024-09-12T02:18:06.460626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:18.877138+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:18.877138+0000","last_peered":"2024-09-12T02:18:18.877138+0000","last_clean":"2024-09-12T02:18:18.877138+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:18.877138+0000","last_undegraded":"2024-09-12T02:18:18.877138+0000","last_fullsized":"2024-09-12T02:18:18.877138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69658,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1058,"num_read_kb":603,"num_write":257,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1091945,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444114,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":501696,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":506632,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:25.750 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:26.050 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:26.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:26 smithi003 ceph-mon[31476]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:26.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:26 smithi003 ceph-mon[31476]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:26.485 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:26 smithi003 ceph-mon[31476]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:26.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 -- 172.21.15.3:0/1034595676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c1008d0 msgr2=0x7ff60c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 --2- 172.21.15.3:0/1034595676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c1008d0 0x7ff60c100cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff608009900 tx=0x7ff60802a410 comp rx=0 tx=0).stop 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 -- 172.21.15.3:0/1034595676 shutdown_connections 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 --2- 172.21.15.3:0/1034595676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c10e480 0x7ff60c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 --2- 172.21.15.3:0/1034595676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c101280 0x7ff60c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 --2- 172.21.15.3:0/1034595676 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c1008d0 0x7ff60c100cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 -- 172.21.15.3:0/1034595676 >> 172.21.15.3:0/1034595676 conn(0x7ff60c0fc7a0 msgr2=0x7ff60c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 -- 172.21.15.3:0/1034595676 shutdown_connections 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.484+0000 7ff6139e5640 1 -- 172.21.15.3:0/1034595676 wait complete. 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 Processor -- start 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 -- start start 2024-09-12T02:18:26.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c1008d0 0x7ff60c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c10e480 0x7ff60c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c114240 con 0x7ff60c101280 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c1140c0 con 0x7ff60c10e480 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff6139e5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c1143c0 con 0x7ff60c1008d0 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff610f80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff610f80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59106/0 (socket says 172.21.15.3:59106) 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff610f80640 1 -- 172.21.15.3:0/162266509 learned_addr learned my addr 172.21.15.3:0/162266509 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff611781640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c1008d0 0x7ff60c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff611f82640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c10e480 0x7ff60c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.485+0000 7ff610f80640 1 -- 172.21.15.3:0/162266509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c1008d0 msgr2=0x7ff60c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff610f80640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c1008d0 0x7ff60c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff610f80640 1 -- 172.21.15.3:0/162266509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c10e480 msgr2=0x7ff60c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff610f80640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c10e480 0x7ff60c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff610f80640 1 -- 172.21.15.3:0/162266509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c19f680 con 0x7ff60c101280 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff610f80640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d950 tx=0x7ff5fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc017070 con 0x7ff60c101280 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c19f970 con 0x7ff60c101280 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc009c30 con 0x7ff60c101280 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00f6b0 con 0x7ff60c101280 2024-09-12T02:18:26.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.486+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1a7250 con 0x7ff60c101280 2024-09-12T02:18:26.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.487+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c100cb0 con 0x7ff60c101280 2024-09-12T02:18:26.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.488+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ff5fc0109d0 con 0x7ff60c101280 2024-09-12T02:18:26.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.489+0000 7ff6027fc640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 0x7ff5f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:26.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.489+0000 7ff611781640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 0x7ff5f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:26.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.489+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff5fc0ced60 con 0x7ff60c101280 2024-09-12T02:18:26.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.489+0000 7ff611781640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 0x7ff5f80a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff608005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:26.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.492+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5fc009de0 con 0x7ff60c101280 2024-09-12T02:18:26.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.706+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff60c19ba30 con 0x7ff60c101280 2024-09-12T02:18:26.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.706+0000 7ff6027fc640 1 -- 172.21.15.3:0/162266509 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7ff60c19ba30 con 0x7ff60c101280 2024-09-12T02:18:26.707 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:26.707 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:18:26.707 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 msgr2=0x7ff5f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 0x7ff5f80a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff608006010 tx=0x7ff608005a40 comp rx=0 tx=0).stop 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 msgr2=0x7ff60c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d950 tx=0x7ff5fc00de10 comp rx=0 tx=0).stop 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 shutdown_connections 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff5f80a2c10 0x7ff5f80a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff60c10e480 0x7ff60c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff60c101280 0x7ff60c19ac10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 --2- 172.21.15.3:0/162266509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60c1008d0 0x7ff60c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:26.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 >> 172.21.15.3:0/162266509 conn(0x7ff60c0fc7a0 msgr2=0x7ff60c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:26.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 shutdown_connections 2024-09-12T02:18:26.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:26.709+0000 7ff6139e5640 1 -- 172.21.15.3:0/162266509 wait complete. 2024-09-12T02:18:26.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:26 smithi026 ceph-mon[35496]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:26.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:26 smithi026 ceph-mon[35496]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:26.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:26 smithi026 ceph-mon[35496]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:26.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:26 smithi092 ceph-mon[36089]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:18:26.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:26 smithi092 ceph-mon[36089]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:26.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:26 smithi092 ceph-mon[36089]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:27.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:27 smithi003 ceph-mon[31476]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:27.474 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:27 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/162266509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:27.507 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:18:27.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:27 smithi026 ceph-mon[35496]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:27.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:27 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/162266509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:27.768 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:27 smithi092 ceph-mon[36089]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:27.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:27 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/162266509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/1942922041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c94102d40 msgr2=0x7f5c94103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/1942922041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c94102d40 0x7f5c94103120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009900 tx=0x7f5c8402a480 comp rx=0 tx=0).stop 2024-09-12T02:18:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/1942922041 shutdown_connections 2024-09-12T02:18:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/1942922041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c9410c170 0x7f5c9410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/1942922041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 0x7f5c9410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/1942922041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c94102d40 0x7f5c94103120 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/1942922041 >> 172.21.15.3:0/1942922041 conn(0x7f5c940fc7a0 msgr2=0x7f5c940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/1942922041 shutdown_connections 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.233+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/1942922041 wait complete. 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 Processor -- start 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 -- start start 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c94102d40 0x7f5c941009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 0x7f5c94100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94111fc0 con 0x7f5c9410c170 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94111e40 con 0x7f5c94103660 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c9a0ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94112140 con 0x7f5c94102d40 2024-09-12T02:18:28.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c93fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59122/0 (socket says 172.21.15.3:59122) 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.234+0000 7f5c93fff640 1 -- 172.21.15.3:0/597915101 learned_addr learned my addr 172.21.15.3:0/597915101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c8bfff640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 0x7f5c94100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c937fe640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c94102d40 0x7f5c941009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 -- 172.21.15.3:0/597915101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c94102d40 msgr2=0x7f5c941009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c94102d40 0x7f5c941009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 -- 172.21.15.3:0/597915101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 msgr2=0x7f5c94100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 0x7f5c94100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 -- 172.21.15.3:0/597915101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c941a6900 con 0x7f5c9410c170 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c93fff640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00eaa0 tx=0x7f5c8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00cdb0 con 0x7f5c9410c170 2024-09-12T02:18:28.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c941a6bf0 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c005020 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0116d0 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.235+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941a7130 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.236+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c94103120 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.238+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5c8c0054d0 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.239+0000 7f5c917fa640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 0x7f5c800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.239+0000 7f5c937fe640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 0x7f5c800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.239+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5c8c0cf6a0 con 0x7f5c9410c170 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.240+0000 7f5c937fe640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 0x7f5c800a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000c00 tx=0x7f5c84005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:28.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.242+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c8c016030 con 0x7f5c9410c170 2024-09-12T02:18:28.419 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:28 smithi003 ceph-mon[31476]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:18:28.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.419+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c94101ed0 con 0x7f5c9410c170 2024-09-12T02:18:28.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.420+0000 7f5c917fa640 1 -- 172.21.15.3:0/597915101 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f5c94101ed0 con 0x7f5c9410c170 2024-09-12T02:18:28.421 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:28.421 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.422+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 msgr2=0x7f5c800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.422+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 0x7f5c800a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000c00 tx=0x7f5c84005b50 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.422+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 msgr2=0x7f5c94101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.422+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00eaa0 tx=0x7f5c8c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 shutdown_connections 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5c800a2c10 0x7f5c800a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5c9410c170 0x7f5c94101420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94103660 0x7f5c94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 --2- 172.21.15.3:0/597915101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c94102d40 0x7f5c941009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 >> 172.21.15.3:0/597915101 conn(0x7f5c940fc7a0 msgr2=0x7f5c940fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:28.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 shutdown_connections 2024-09-12T02:18:28.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:28.423+0000 7f5c9a0ad640 1 -- 172.21.15.3:0/597915101 wait complete. 2024-09-12T02:18:28.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:28 smithi026 ceph-mon[35496]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:18:28.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:28 smithi092 ceph-mon[36089]: pgmap v338: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:18:29.124 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T02:18:29.124 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T02:18:29.137 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:18:29.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T02:18:29.138 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T02:18:29.138 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:18:29.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:18:29.141 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T02:18:29.153 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:18:29.153 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:18:29.419 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:29.423 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:29.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:29 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/597915101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:29.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:29 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/597915101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:29.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:29 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/597915101' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 -- 172.21.15.3:0/3479638030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c106f10 msgr2=0x7fd61c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 --2- 172.21.15.3:0/3479638030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c106f10 0x7fd61c10ea90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd61402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 -- 172.21.15.3:0/3479638030 shutdown_connections 2024-09-12T02:18:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 --2- 172.21.15.3:0/3479638030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c106f10 0x7fd61c10ea90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 --2- 172.21.15.3:0/3479638030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c102790 0x7fd61c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 --2- 172.21.15.3:0/3479638030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c101de0 0x7fd61c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 -- 172.21.15.3:0/3479638030 >> 172.21.15.3:0/3479638030 conn(0x7fd61c0fd8c0 msgr2=0x7fd61c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 -- 172.21.15.3:0/3479638030 shutdown_connections 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.071+0000 7fd621610640 1 -- 172.21.15.3:0/3479638030 wait complete. 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 Processor -- start 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 -- start start 2024-09-12T02:18:30.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 0x7fd61c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c106f10 0x7fd61c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c111fc0 con 0x7fd61c102790 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c111e40 con 0x7fd61c106f10 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd621610640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c112140 con 0x7fd61c101de0 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd61a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd61a7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57186/0 (socket says 172.21.15.3:57186) 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.072+0000 7fd61a7fc640 1 -- 172.21.15.3:0/481438946 learned_addr learned my addr 172.21.15.3:0/481438946 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c106f10 0x7fd61c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61affd640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 0x7fd61c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 -- 172.21.15.3:0/481438946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 msgr2=0x7fd61c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 0x7fd61c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 -- 172.21.15.3:0/481438946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c106f10 msgr2=0x7fd61c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c106f10 0x7fd61c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 -- 172.21.15.3:0/481438946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c19d350 con 0x7fd61c102790 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61affd640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 0x7fd61c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd61000e9f0 tx=0x7fd61000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd603fff640 1 -- 172.21.15.3:0/481438946 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61000cd80 con 0x7fd61c102790 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c19d640 con 0x7fd61c102790 2024-09-12T02:18:30.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd603fff640 1 -- 172.21.15.3:0/481438946 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd610005500 con 0x7fd61c102790 2024-09-12T02:18:30.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd603fff640 1 -- 172.21.15.3:0/481438946 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610010690 con 0x7fd61c102790 2024-09-12T02:18:30.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.073+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a4f20 con 0x7fd61c102790 2024-09-12T02:18:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd603fff640 1 -- 172.21.15.3:0/481438946 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd6100046c0 con 0x7fd61c102790 2024-09-12T02:18:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 Processor -- start 2024-09-12T02:18:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 -- start start 2024-09-12T02:18:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fb7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 0x55fc24fba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fc24f12280 con 0x55fc24facb60 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc24eb9b10 con 0x55fc24fad480 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd6239e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc24f0e240 con 0x55fc24fb7d40 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57192/0 (socket says 172.21.15.3:57192) 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.076+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2590590003 learned_addr learned my addr 172.21.15.3:0/2590590003 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61affd640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fb7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd603fff640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 0x7fd6040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 0x55fc24fba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2590590003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 msgr2=0x55fc24fba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd603fff640 1 -- 172.21.15.3:0/481438946 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6100cf630 con 0x7fd61c102790 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 0x55fc24fba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2590590003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 msgr2=0x55fc24fb7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fb7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2590590003 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc24ec61f0 con 0x55fc24facb60 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 0x55fc24fba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 0x7fd6040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd61402aeb0 tx=0x7fd61402f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc3364e95393387 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd6017fa640 1 -- 172.21.15.3:0/2590590003 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614042040 con 0x55fc24facb60 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd6017fa640 1 -- 172.21.15.3:0/2590590003 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614036070 con 0x55fc24facb60 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd6017fa640 1 -- 172.21.15.3:0/2590590003 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61402f830 con 0x55fc24facb60 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.077+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 0x7fd6040a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fe0 tx=0x7fd61404e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 -- 172.21.15.3:0/2590590003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 msgr2=0x55fc24facf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd61402aeb0 tx=0x7fd61402f4b0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 -- 172.21.15.3:0/2590590003 shutdown_connections 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fb7d40 0x55fc24fba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fb7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2590590003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24facf40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 -- 172.21.15.3:0/2590590003 >> 172.21.15.3:0/2590590003 conn(0x55fc24fa90f0 msgr2=0x55fc24fa94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 -- 172.21.15.3:0/2590590003 shutdown_connections 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.078+0000 7fd6239e9740 1 -- 172.21.15.3:0/2590590003 wait complete. 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 -- start start 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 Processor -- start 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24facb60 0x55fc24fb4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55fc24fba910 con 0x55fc24fb7d40 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55fc24eccf10 con 0x55fc24facb60 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55fc24fbaa90 con 0x55fc24fad480 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd6239e9740 10 client.0 _ll_register_callbacks cb 0x55fc24e95a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57206/0 (socket says 172.21.15.3:57206) 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2965323702 learned_addr learned my addr 172.21.15.3:0/2965323702 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61affd640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24facb60 0x55fc24fb4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 msgr2=0x55fc24fb5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2965323702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24facb60 msgr2=0x55fc24fb4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24facb60 0x55fc24fb4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.079+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2965323702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc24c836a0 con 0x55fc24fb7d40 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd614005e30 tx=0x7fd614004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb7bc1a085e0b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614059070 con 0x55fc24fb7d40 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd61402fdc0 con 0x55fc24fb7d40 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd614036070 con 0x55fc24fb7d40 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd614065050 con 0x55fc24fb7d40 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 10 client.15303 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:30.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc24fba910 con 0x55fc24fb7d40 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd614036070 con 0x55fc24fb7d40 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd614030650 con 0x55fc24fb7d40 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 10 client.15303 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 10 client.15303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 20 client.15303 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc24fc75b0 0x55fc24fc99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 4 client.15303 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.080+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc24f1f7f0 con 0x55fc24fc75b0 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.081+0000 7fd61affd640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc24fc75b0 0x55fc24fc99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.081+0000 7fd61affd640 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc24fc75b0 0x55fc24fc99a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6438b1b3feeafc server_cookie=268b8db90ddd4d50 in_seq=0 out_seq=0 2024-09-12T02:18:30.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.081+0000 7fd5feff5640 10 client.15303 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd5feff5640 1 -- 172.21.15.3:0/2965323702 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc24f1f7f0 con 0x55fc24fc75b0 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd5feff5640 10 client.15303 handle_command_reply: tid=0 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 client.15303 shutdown 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 msgr2=0x55fc24fb5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd614005e30 tx=0x7fd614004630 comp rx=0 tx=0).stop 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 shutdown_connections 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc24fc75b0 0x55fc24fc99a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fb5650 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2965323702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24facb60 0x55fc24fb4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd5feff5640 0 client.15303 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 >> 172.21.15.3:0/2965323702 conn(0x55fc24fa90f0 msgr2=0x55fc24faee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 shutdown_connections 2024-09-12T02:18:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 1 -- 172.21.15.3:0/2965323702 wait complete. 2024-09-12T02:18:30.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.082+0000 7fd6239e9740 20 client.15303 trim_cache size 0 max 16384 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.099+0000 7fd6239e9740 1 Processor -- start 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 -- start start 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 0x55fc24fcf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ad510 con 0x55fc24facb60 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1af050 con 0x55fc24fb7d40 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd6239e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1adbe0 con 0x55fc24fad480 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57220/0 (socket says 172.21.15.3:57220) 2024-09-12T02:18:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 -- 172.21.15.3:0/3712240664 learned_addr learned my addr 172.21.15.3:0/3712240664 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 0x55fc24fcf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 -- 172.21.15.3:0/3712240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 msgr2=0x55fc24fb3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 -- 172.21.15.3:0/3712240664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 msgr2=0x55fc24fcf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 0x55fc24fcf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 -- 172.21.15.3:0/3712240664 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc24fa8f20 con 0x55fc24facb60 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 0x55fc24fcf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.100+0000 7fd61affd640 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be119d16747cbc28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd600ff9640 1 -- 172.21.15.3:0/3712240664 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c005020 con 0x55fc24facb60 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd600ff9640 1 -- 172.21.15.3:0/3712240664 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c0047a0 con 0x55fc24facb60 2024-09-12T02:18:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd600ff9640 1 -- 172.21.15.3:0/3712240664 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01b6c0 con 0x55fc24facb60 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- 172.21.15.3:0/3712240664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 msgr2=0x55fc24faf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd60c000c00 tx=0x7fd60c012820 comp rx=0 tx=0).stop 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- 172.21.15.3:0/3712240664 shutdown_connections 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fb7d40 0x55fc24fcf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24fad480 0x55fc24fb3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 --2- 172.21.15.3:0/3712240664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24facb60 0x55fc24faf5c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- 172.21.15.3:0/3712240664 >> 172.21.15.3:0/3712240664 conn(0x55fc24fa90f0 msgr2=0x55fc24fb3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- 172.21.15.3:0/3712240664 shutdown_connections 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- 172.21.15.3:0/3712240664 wait complete. 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.101+0000 7fd6239e9740 1 -- start start 2024-09-12T02:18:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 Processor -- start 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 0x55fc24fbbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fbc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ae980 con 0x55fc24fb7d40 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ae2b0 con 0x55fc24fad480 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ab4f0 con 0x55fc24facb60 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd6239e9740 10 client.0 _ll_register_callbacks cb 0x7fd60c00c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57234/0 (socket says 172.21.15.3:57234) 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2808816481 learned_addr learned my addr 172.21.15.3:0/2808816481 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61affd640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 0x55fc24fbbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fbc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 msgr2=0x55fc24fbbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 0x55fc24fbbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2808816481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 msgr2=0x55fc24fbc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fbc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61b7fe640 1 -- 172.21.15.3:0/2808816481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc25031110 con 0x55fc24fb7d40 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61affd640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 0x55fc24fbbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.102+0000 7fd61a7fc640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fbc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd61b7fe640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd614002230 tx=0x7fd6140048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326a32d56ab6a93a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614062070 con 0x55fc24fb7d40 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6140307c0 con 0x55fc24fb7d40 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c0047a0 con 0x55fc24fb7d40 2024-09-12T02:18:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd614070050 con 0x55fc24fb7d40 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 10 client.15309 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc250323a0 con 0x55fc24fb7d40 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c0047a0 con 0x55fc24fb7d40 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6140316a0 con 0x55fc24fb7d40 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 10 client.15309 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 10 client.15309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 20 client.15309 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc25032a00 0x55fc25034df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 4 client.15309 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.103+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fc24f19d00 con 0x55fc25032a00 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.104+0000 7fd61affd640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc25032a00 0x55fc25034df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.104+0000 7fd61affd640 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc25032a00 0x55fc25034df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d23dc26f875 server_cookie=1c9a14861d0b215f in_seq=0 out_seq=0 2024-09-12T02:18:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.104+0000 7fd5feff5640 10 client.15309 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd5feff5640 1 -- 172.21.15.3:0/2808816481 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fc24f19d00 con 0x55fc25032a00 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd5feff5640 10 client.15309 handle_command_reply: tid=0 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 client.15309 shutdown 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 msgr2=0x55fc24fbc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd614002230 tx=0x7fd6140048d0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 shutdown_connections 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55fc25032a00 0x55fc25034df0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55fc24fb7d40 0x55fc24fbc570 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fc24fad480 0x55fc24fbc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd5feff5640 0 client.15309 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 --2- 172.21.15.3:0/2808816481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55fc24facb60 0x55fc24fbbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 >> 172.21.15.3:0/2808816481 conn(0x55fc24fa90f0 msgr2=0x55fc24fa8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.105+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 shutdown_connections 2024-09-12T02:18:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd6239e9740 1 -- 172.21.15.3:0/2808816481 wait complete. 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd6239e9740 20 client.15309 trim_cache size 0 max 16384 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "d636ed9c-2055-43ca-a95c-a1dc6cab5ea1", 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-12T02:18:30.111 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 msgr2=0x7fd6040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 0x7fd6040a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fe0 tx=0x7fd61404e000 comp rx=0 tx=0).stop 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 msgr2=0x7fd61c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd61000e9f0 tx=0x7fd61000edf0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 shutdown_connections 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6040a2c10 0x7fd6040a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd61c106f10 0x7fd61c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd61c102790 0x7fd61c1988e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 --2- 172.21.15.3:0/481438946 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd61c101de0 0x7fd61c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 >> 172.21.15.3:0/481438946 conn(0x7fd61c0fd8c0 msgr2=0x7fd61c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 shutdown_connections 2024-09-12T02:18:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.106+0000 7fd621610640 1 -- 172.21.15.3:0/481438946 wait complete. 2024-09-12T02:18:30.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 -- 172.21.15.3:0/3524051414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40100a80 msgr2=0x7fcd40100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3524051414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40100a80 0x7fcd40100ee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009930 tx=0x7fcd3002a420 comp rx=0 tx=0).stop 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 -- 172.21.15.3:0/3524051414 shutdown_connections 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3524051414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd4010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3524051414 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40100a80 0x7fcd40100ee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3524051414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40106a80 0x7fcd40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 -- 172.21.15.3:0/3524051414 >> 172.21.15.3:0/3524051414 conn(0x7fcd400fc7a0 msgr2=0x7fcd400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 -- 172.21.15.3:0/3524051414 shutdown_connections 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.283+0000 7fcd47df8640 1 -- 172.21.15.3:0/3524051414 wait complete. 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 Processor -- start 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 -- start start 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 0x7fcd401992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd40199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40110e40 con 0x7fcd40106a80 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40110cc0 con 0x7fcd40101420 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd47df8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcd40110fc0 con 0x7fcd40100a80 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd46395640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd46395640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57254/0 (socket says 172.21.15.3:57254) 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd46395640 1 -- 172.21.15.3:0/3730022186 learned_addr learned my addr 172.21.15.3:0/3730022186 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd45393640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd40199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd45b94640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 0x7fcd401992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd46395640 1 -- 172.21.15.3:0/3730022186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 msgr2=0x7fcd401992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.284+0000 7fcd46395640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 0x7fcd401992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd46395640 1 -- 172.21.15.3:0/3730022186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 msgr2=0x7fcd40199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd46395640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd40199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd46395640 1 -- 172.21.15.3:0/3730022186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd401a4890 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd45393640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd40199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd45b94640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 0x7fcd401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd46395640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800eb10 tx=0x7fcd3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004b20 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd38004cc0 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd401a4a60 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38010760 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.285+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd401a4f20 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.286+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd40100ee0 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.287+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fcd38005020 con 0x7fcd40106a80 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.288+0000 7fcd36ffd640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 0x7fcd2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.288+0000 7fcd45b94640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 0x7fcd2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:30.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.288+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fcd380cf8c0 con 0x7fcd40106a80 2024-09-12T02:18:30.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.288+0000 7fcd45b94640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 0x7fcd2c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009ab0 tx=0x7fcd3c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:30.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.292+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd3809c6b0 con 0x7fcd40106a80 2024-09-12T02:18:30.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.477+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd40109080 con 0x7fcd2c0a2c10 2024-09-12T02:18:30.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.483+0000 7fcd36ffd640 1 -- 172.21.15.3:0/3730022186 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259113 (secure 0 0 0) 0x7fcd3c00ea30 con 0x7fcd2c0a2c10 2024-09-12T02:18:30.486 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:30.487 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 msgr2=0x7fcd2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 0x7fcd2c0a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009ab0 tx=0x7fcd3c0094b0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 msgr2=0x7fcd401a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800eb10 tx=0x7fcd3800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 shutdown_connections 2024-09-12T02:18:30.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcd2c0a2c10 0x7fcd2c0a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcd40106a80 0x7fcd401a4350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd40101420 0x7fcd40199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 --2- 172.21.15.3:0/3730022186 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd40100a80 0x7fcd401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 >> 172.21.15.3:0/3730022186 conn(0x7fcd400fc7a0 msgr2=0x7fcd400fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 shutdown_connections 2024-09-12T02:18:30.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:30.486+0000 7fcd47df8640 1 -- 172.21.15.3:0/3730022186 wait complete. 2024-09-12T02:18:30.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:30 smithi026 ceph-mon[35496]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:18:30.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:30 smithi003 ceph-mon[31476]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:18:30.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:30 smithi092 ceph-mon[36089]: pgmap v339: 97 pgs: 97 active+clean; 650 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:18:31.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:18:31.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:31 smithi003 ceph-mon[31476]: scrub queued for path: / 2024-09-12T02:18:31.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:31 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:31 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:18:31.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:31 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:31 smithi026 ceph-mon[35496]: scrub queued for path: / 2024-09-12T02:18:31.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:31 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:31 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:18:31.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:31 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.787 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-12T02:18:30.459057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667795,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1263,"num_read_kb":767,"num_write":449,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360456,"kb_used_data":11208,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581368,"statfs":{"total":1151940427776,"available":1151571320832,"internally_reserved":0,"allocated":11476992,"data_stored":7253541,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8236,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82,"num_write":16,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002812"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451322+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451322+0000","last_peered":"2024-09-12T02:18:06.451322+0000","last_clean":"2024-09-12T02:18:06.451322+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451322+0000","last_undegraded":"2024-09-12T02:18:06.451322+0000","last_fullsized":"2024-09-12T02:18:06.451322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:25.887845+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:25.887845+0000","last_peered":"2024-09-12T02:18:25.887845+0000","last_clean":"2024-09-12T02:18:25.887845+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:25.887845+0000","last_undegraded":"2024-09-12T02:18:25.887845+0000","last_fullsized":"2024-09-12T02:18:25.887845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":246,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891273+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:28.891273+0000","last_peered":"2024-09-12T02:18:28.891273+0000","last_clean":"2024-09-12T02:18:28.891273+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:28.891273+0000","last_undegraded":"2024-09-12T02:18:28.891273+0000","last_fullsized":"2024-09-12T02:18:28.891273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'40","reported_seq":294,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891246+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:28.891246+0000","last_peered":"2024-09-12T02:18:28.891246+0000","last_clean":"2024-09-12T02:18:28.891246+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:28.891246+0000","last_undegraded":"2024-09-12T02:18:28.891246+0000","last_fullsized":"2024-09-12T02:18:28.891246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69536,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":184,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:25.888008+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:25.888008+0000","last_peered":"2024-09-12T02:18:25.888008+0000","last_clean":"2024-09-12T02:18:25.888008+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:25.888008+0000","last_undegraded":"2024-09-12T02:18:25.888008+0000","last_fullsized":"2024-09-12T02:18:25.888008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:24.886345+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:24.886345+0000","last_peered":"2024-09-12T02:18:24.886345+0000","last_clean":"2024-09-12T02:18:24.886345+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:24.886345+0000","last_undegraded":"2024-09-12T02:18:24.886345+0000","last_fullsized":"2024-09-12T02:18:24.886345+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747039+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:39.747039+0000","last_peered":"2024-09-12T02:16:39.747039+0000","last_clean":"2024-09-12T02:16:39.747039+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:39.747039+0000","last_undegraded":"2024-09-12T02:16:39.747039+0000","last_fullsized":"2024-09-12T02:16:39.747039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:26.888577+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:26.888577+0000","last_peered":"2024-09-12T02:18:26.888577+0000","last_clean":"2024-09-12T02:18:26.888577+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:26.888577+0000","last_undegraded":"2024-09-12T02:18:26.888577+0000","last_fullsized":"2024-09-12T02:18:26.888577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447475+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447475+0000","last_peered":"2024-09-12T02:18:06.447475+0000","last_clean":"2024-09-12T02:18:06.447475+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447475+0000","last_undegraded":"2024-09-12T02:18:06.447475+0000","last_fullsized":"2024-09-12T02:18:06.447475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:26.888498+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:26.888498+0000","last_peered":"2024-09-12T02:18:26.888498+0000","last_clean":"2024-09-12T02:18:26.888498+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:26.888498+0000","last_undegraded":"2024-09-12T02:18:26.888498+0000","last_fullsized":"2024-09-12T02:18:26.888498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'17","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.137207+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:18:21.137207+0000","last_peered":"2024-09-12T02:18:21.137207+0000","last_clean":"2024-09-12T02:18:21.137207+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:18:21.137207+0000","last_undegraded":"2024-09-12T02:18:21.137207+0000","last_fullsized":"2024-09-12T02:18:21.137207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157535+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.157535+0000","last_peered":"2024-09-12T02:16:38.157535+0000","last_clean":"2024-09-12T02:16:38.157535+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.157535+0000","last_undegraded":"2024-09-12T02:16:38.157535+0000","last_fullsized":"2024-09-12T02:16:38.157535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460626+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.460626+0000","last_peered":"2024-09-12T02:18:06.460626+0000","last_clean":"2024-09-12T02:18:06.460626+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.460626+0000","last_undegraded":"2024-09-12T02:18:06.460626+0000","last_fullsized":"2024-09-12T02:18:06.460626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.891913+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:29.891913+0000","last_peered":"2024-09-12T02:18:29.891913+0000","last_clean":"2024-09-12T02:18:29.891913+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:29.891913+0000","last_undegraded":"2024-09-12T02:18:29.891913+0000","last_fullsized":"2024-09-12T02:18:29.891913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1137,"num_read_kb":658,"num_write":266,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":231579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1100181,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790019,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":506711,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":509545,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:31.791 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:18:31.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:31 smithi092 ceph-mon[36089]: scrub queued for path: / 2024-09-12T02:18:31.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:31 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:31 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:18:31.792 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:31 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:18:31.834 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- 172.21.15.3:0/4274267522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8101280 msgr2=0x7ff7e810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/4274267522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8101280 0x7ff7e810df40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009900 tx=0x7ff7d002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- 172.21.15.3:0/4274267522 shutdown_connections 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/4274267522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e810e480 0x7ff7e8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/4274267522 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e8101280 0x7ff7e810df40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/4274267522 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 0x7ff7e8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- 172.21.15.3:0/4274267522 >> 172.21.15.3:0/4274267522 conn(0x7ff7e80fc7a0 msgr2=0x7ff7e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- 172.21.15.3:0/4274267522 shutdown_connections 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- 172.21.15.3:0/4274267522 wait complete. 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 Processor -- start 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.289+0000 7ff7edcc9640 1 -- start start 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 0x7ff7e819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e8101280 0x7ff7e819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81141b0 con 0x7ff7e810e480 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8114030 con 0x7ff7e8101280 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7edcc9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8114330 con 0x7ff7e81008d0 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57270/0 (socket says 172.21.15.3:57270) 2024-09-12T02:18:32.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1807666507 learned_addr learned my addr 172.21.15.3:0/1807666507 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 0x7ff7e819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7deffd640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e8101280 0x7ff7e819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1807666507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 msgr2=0x7ff7e819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 0x7ff7e819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1807666507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e8101280 msgr2=0x7ff7e819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e8101280 0x7ff7e819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 -- 172.21.15.3:0/1807666507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e819f6a0 con 0x7ff7e810e480 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.290+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009fd0 tx=0x7ff7e000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.291+0000 7ff7e57fa640 1 -- 172.21.15.3:0/1807666507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0004d20 con 0x7ff7e810e480 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.291+0000 7ff7e57fa640 1 -- 172.21.15.3:0/1807666507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e0005020 con 0x7ff7e810e480 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.291+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e819f990 con 0x7ff7e810e480 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.291+0000 7ff7e57fa640 1 -- 172.21.15.3:0/1807666507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00116d0 con 0x7ff7e810e480 2024-09-12T02:18:32.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.291+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e81a7270 con 0x7ff7e810e480 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7e57fa640 1 -- 172.21.15.3:0/1807666507 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ff7e00051d0 con 0x7ff7e810e480 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 Processor -- start 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 -- start start 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96f4a0 0x56519c96f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c97a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56519c8d52e0 con 0x56519c97a680 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56519c87cce0 con 0x56519c96fdc0 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.293+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56519c8d12a0 con 0x56519c96f4a0 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57276/0 (socket says 172.21.15.3:57276) 2024-09-12T02:18:32.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 -- 172.21.15.3:0/3637444074 learned_addr learned my addr 172.21.15.3:0/3637444074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c97a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e57fa640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 0x7ff7d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96f4a0 0x56519c96f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 0x7ff7d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 -- 172.21.15.3:0/3637444074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96f4a0 msgr2=0x56519c96f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96f4a0 0x56519c96f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 -- 172.21.15.3:0/3637444074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 msgr2=0x56519c97a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c97a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e57fa640 1 -- 172.21.15.3:0/1807666507 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff7e00cf740 con 0x7ff7e810e480 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 -- 172.21.15.3:0/3637444074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519c861440 con 0x56519c97a680 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c97a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7deffd640 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0031550 tx=0x7ff7d0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4037abd30d263d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.294+0000 7ff7e7fff640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 0x7ff7d40a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7e009bb30 tx=0x7ff7e000ef90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7ddffb640 1 -- 172.21.15.3:0/3637444074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0035040 con 0x56519c97a680 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7ddffb640 1 -- 172.21.15.3:0/3637444074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d00035f0 con 0x56519c97a680 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7ddffb640 1 -- 172.21.15.3:0/3637444074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d003d6b0 con 0x56519c97a680 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 -- 172.21.15.3:0/3637444074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 msgr2=0x56519c97ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0031550 tx=0x7ff7d0005b50 comp rx=0 tx=0).stop 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 -- 172.21.15.3:0/3637444074 shutdown_connections 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c97a680 0x56519c97ca70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c97a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/3637444074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96f4a0 0x56519c96f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 -- 172.21.15.3:0/3637444074 >> 172.21.15.3:0/3637444074 conn(0x56519c96ba30 msgr2=0x56519c96be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 -- 172.21.15.3:0/3637444074 shutdown_connections 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.295+0000 7ff7f00a2740 1 -- 172.21.15.3:0/3637444074 wait complete. 2024-09-12T02:18:32.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 -- start start 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 Processor -- start 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c979f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 0x56519c977740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56519c97d250 con 0x56519c96fdc0 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56519c890050 con 0x56519c96f4a0 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56519c97d3d0 con 0x56519c97a680 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.296+0000 7ff7f00a2740 10 client.0 _ll_register_callbacks cb 0x56519c8d0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57290/0 (socket says 172.21.15.3:57290) 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2602636438 learned_addr learned my addr 172.21.15.3:0/2602636438 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c979f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 0x56519c977740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 msgr2=0x56519c977740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 0x56519c977740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2602636438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 msgr2=0x56519c979f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c979f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2602636438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519c6463c0 con 0x56519c96fdc0 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 0x56519c977740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012740 tx=0x7ff7d8012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2eab91f349aab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004050 con 0x56519c96fdc0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff7d80041f0 con 0x56519c96fdc0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.297+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d00035f0 con 0x56519c96fdc0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7d80043f0 con 0x56519c96fdc0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 10 client.15327 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56519c988c50 con 0x56519c96fdc0 2024-09-12T02:18:32.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d8012e50 con 0x56519c96fdc0 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7d8010430 con 0x56519c96fdc0 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 10 client.15327 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 10 client.15327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 20 client.15327 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c98a110 0x56519c98c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 4 client.15327 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.298+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff7d00035f0 con 0x56519c98a110 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.299+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c98a110 0x56519c98c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.299+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c98a110 0x56519c98c500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897dc5bb9a8c7346 server_cookie=e683b4b6078f1d32 in_seq=0 out_seq=0 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.299+0000 7ff7c6ffd640 10 client.15327 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/2602636438 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff7d00035f0 con 0x56519c98a110 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7c6ffd640 10 client.15327 handle_command_reply: tid=0 2024-09-12T02:18:32.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 client.15327 shutdown 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 msgr2=0x56519c977200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012740 tx=0x7ff7d8012c00 comp rx=0 tx=0).stop 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 shutdown_connections 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c98a110 0x56519c98c500 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c97a680 0x56519c977740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c977200 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2602636438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c979f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7c6ffd640 0 client.15327 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 >> 172.21.15.3:0/2602636438 conn(0x56519c96ba30 msgr2=0x56519c971710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 shutdown_connections 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2602636438 wait complete. 2024-09-12T02:18:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.300+0000 7ff7f00a2740 20 client.15327 trim_cache size 0 max 16384 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.318+0000 7ff7f00a2740 1 Processor -- start 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.318+0000 7ff7f00a2740 1 -- start start 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c9864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c9727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81af770 con 0x56519c96f4a0 2024-09-12T02:18:32.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b12b0 con 0x56519c96fdc0 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81afe40 con 0x56519c96ba30 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57296/0 (socket says 172.21.15.3:57296) 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2629077811 learned_addr learned my addr 172.21.15.3:0/2629077811 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c9864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7deffd640 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c9727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2629077811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 msgr2=0x56519c9864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c9864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2629077811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 msgr2=0x56519c9727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c9727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.319+0000 7ff7e77fe640 1 -- 172.21.15.3:0/2629077811 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519c97a2c0 con 0x56519c96f4a0 2024-09-12T02:18:32.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.320+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012420 tx=0x7ff7d80096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212916aa9c7a7219 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.320+0000 7ff7dd7fa640 1 -- 172.21.15.3:0/2629077811 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8010430 con 0x56519c96f4a0 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.320+0000 7ff7dd7fa640 1 -- 172.21.15.3:0/2629077811 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d8012e50 con 0x56519c96f4a0 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.320+0000 7ff7dd7fa640 1 -- 172.21.15.3:0/2629077811 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8005360 con 0x56519c96f4a0 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2629077811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 msgr2=0x56519c9771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8012420 tx=0x7ff7d80096d0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2629077811 shutdown_connections 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96fdc0 0x56519c9727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96f4a0 0x56519c9771c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/2629077811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c9864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2629077811 >> 172.21.15.3:0/2629077811 conn(0x56519c9778a0 msgr2=0x56519c972090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2629077811 shutdown_connections 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- 172.21.15.3:0/2629077811 wait complete. 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.321+0000 7ff7f00a2740 1 -- start start 2024-09-12T02:18:32.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 Processor -- start 2024-09-12T02:18:32.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c97e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c97e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0be0 con 0x56519c96fdc0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81b0510 con 0x56519c96f4a0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e81ad750 con 0x56519c96ba30 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7f00a2740 10 client.0 _ll_register_callbacks cb 0x7ff7d8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7deffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7deffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57304/0 (socket says 172.21.15.3:57304) 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7deffd640 1 -- 172.21.15.3:0/310975236 learned_addr learned my addr 172.21.15.3:0/310975236 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.322+0000 7ff7deffd640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c97e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 -- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 msgr2=0x56519c97e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c97e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 -- 172.21.15.3:0/310975236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 msgr2=0x56519c97e940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c97e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 -- 172.21.15.3:0/310975236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519c9f3e20 con 0x56519c96fdc0 2024-09-12T02:18:32.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7deffd640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0006010 tx=0x7ff7d8009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b341c314782554 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0035040 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff7d0004070 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.323+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d8012e50 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7d0004270 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 10 client.15333 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56519c9fd920 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d8012e50 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7d0068070 con 0x56519c96fdc0 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 10 client.15333 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 10 client.15333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 20 client.15333 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c97a680 0x56519c9ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:32.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 4 client.15333 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.324+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56519c8e2850 con 0x56519c97a680 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c97a680 0x56519c9ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7c6ffd640 10 client.15333 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7e77fe640 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c97a680 0x56519c9ffb70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9050e43b1dc582 server_cookie=13483c11099e6558 in_seq=0 out_seq=0 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7c6ffd640 1 -- 172.21.15.3:0/310975236 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56519c8e2850 con 0x56519c97a680 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7c6ffd640 10 client.15333 handle_command_reply: tid=0 2024-09-12T02:18:32.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.325+0000 7ff7f00a2740 1 client.15333 shutdown 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 msgr2=0x56519c97ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0006010 tx=0x7ff7d8009e90 comp rx=0 tx=0).stop 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 shutdown_connections 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56519c97a680 0x56519c9ffb70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56519c96fdc0 0x56519c97ee80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56519c96f4a0 0x56519c97e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 --2- 172.21.15.3:0/310975236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56519c96ba30 0x56519c97e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 >> 172.21.15.3:0/310975236 conn(0x56519c9778a0 msgr2=0x56519c9727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7c6ffd640 0 client.15333 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 shutdown_connections 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 1 -- 172.21.15.3:0/310975236 wait complete. 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.326+0000 7ff7f00a2740 20 client.15333 trim_cache size 0 max 16384 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout: "d636ed9c-2055-43ca-a95c-a1dc6cab5ea1": { 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-12T02:18:32.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "d636ed9c-2055-43ca-a95c-a1dc6cab5ea1", 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 msgr2=0x7ff7d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 0x7ff7d40a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7e009bb30 tx=0x7ff7e000ef90 comp rx=0 tx=0).stop 2024-09-12T02:18:32.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 msgr2=0x7ff7e819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009fd0 tx=0x7ff7e000ee60 comp rx=0 tx=0).stop 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 shutdown_connections 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff7d40a2c10 0x7ff7d40a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7e810e480 0x7ff7e819efc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7e8101280 0x7ff7e819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 --2- 172.21.15.3:0/1807666507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7e81008d0 0x7ff7e819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 >> 172.21.15.3:0/1807666507 conn(0x7ff7e80fc7a0 msgr2=0x7ff7e80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 shutdown_connections 2024-09-12T02:18:32.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:32.327+0000 7ff7edcc9640 1 -- 172.21.15.3:0/1807666507 wait complete. 2024-09-12T02:18:32.336 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:32.641 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:32 smithi003 ceph-mon[31476]: pgmap v340: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T02:18:32.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:32 smithi003 ceph-mon[31476]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:32.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:32 smithi026 ceph-mon[35496]: pgmap v340: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T02:18:32.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:32 smithi026 ceph-mon[35496]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:32.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:32 smithi092 ceph-mon[36089]: pgmap v340: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T02:18:32.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:32 smithi092 ceph-mon[36089]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:32.974 INFO:tasks.cephfs.filesystem:scrub status for tag:d636ed9c-2055-43ca-a95c-a1dc6cab5ea1 - {'path': '/', 'tag': 'd636ed9c-2055-43ca-a95c-a1dc6cab5ea1', 'options': 'recursive,force'} 2024-09-12T02:18:33.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.453+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2666027130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0102ef0 msgr2=0x7fc2f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:33.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.453+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2666027130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0102ef0 0x7fc2f010da40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009900 tx=0x7fc2e802a480 comp rx=0 tx=0).stop 2024-09-12T02:18:33.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2666027130 shutdown_connections 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2666027130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0102ef0 0x7fc2f010da40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2666027130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102570 0x7fc2f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2666027130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0100e20 0x7fc2f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2666027130 >> 172.21.15.3:0/2666027130 conn(0x7fc2f00fc7d0 msgr2=0x7fc2f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2666027130 shutdown_connections 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2666027130 wait complete. 2024-09-12T02:18:33.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 Processor -- start 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.454+0000 7fc2f84e1640 1 -- start start 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 0x7fc2f00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102ef0 0x7fc2f00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0110f80 con 0x7fc2f0100e20 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0110e00 con 0x7fc2f0102ef0 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f84e1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0111100 con 0x7fc2f0102570 2024-09-12T02:18:33.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57320/0 (socket says 172.21.15.3:57320) 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 -- 172.21.15.3:0/2695333531 learned_addr learned my addr 172.21.15.3:0/2695333531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f6a7e640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102ef0 0x7fc2f00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f5a7c640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 0x7fc2f00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 -- 172.21.15.3:0/2695333531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 msgr2=0x7fc2f00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 0x7fc2f00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 -- 172.21.15.3:0/2695333531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102ef0 msgr2=0x7fc2f00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102ef0 0x7fc2f00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 -- 172.21.15.3:0/2695333531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f01a6930 con 0x7fc2f0100e20 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f5a7c640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 0x7fc2f00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.455+0000 7fc2f627d640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00ea80 tx=0x7fc2ec00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.456+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00ce40 con 0x7fc2f0100e20 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.456+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec005020 con 0x7fc2f0100e20 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.456+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01a6c20 con 0x7fc2f0100e20 2024-09-12T02:18:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.456+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec011760 con 0x7fc2f0100e20 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.456+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01a7110 con 0x7fc2f0100e20 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.457+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0101200 con 0x7fc2f0100e20 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.458+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc2ec0051d0 con 0x7fc2f0100e20 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.459+0000 7fc2e77fe640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 0x7fc2dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.459+0000 7fc2f5a7c640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 0x7fc2dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.459+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc2ec0cf800 con 0x7fc2f0100e20 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.459+0000 7fc2f5a7c640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 0x7fc2dc0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005fd0 tx=0x7fc2e0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:33.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.463+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2ec016030 con 0x7fc2f0100e20 2024-09-12T02:18:33.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.638+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2f0062630 con 0x7fc2dc0a2c10 2024-09-12T02:18:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.644+0000 7fc2e77fe640 1 -- 172.21.15.3:0/2695333531 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259113 (secure 0 0 0) 0x7fc2f0062630 con 0x7fc2dc0a2c10 2024-09-12T02:18:33.647 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:33.648 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:33.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 msgr2=0x7fc2dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 0x7fc2dc0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005fd0 tx=0x7fc2e0005850 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 msgr2=0x7fc2f00fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00ea80 tx=0x7fc2ec00ef40 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 shutdown_connections 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc2dc0a2c10 0x7fc2dc0a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f0102ef0 0x7fc2f00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2f0102570 0x7fc2f00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 --2- 172.21.15.3:0/2695333531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc2f0100e20 0x7fc2f00fcc80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:33.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.647+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 >> 172.21.15.3:0/2695333531 conn(0x7fc2f00fc7d0 msgr2=0x7fc2f006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:33.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.648+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 shutdown_connections 2024-09-12T02:18:33.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:33.648+0000 7fc2f84e1640 1 -- 172.21.15.3:0/2695333531 wait complete. 2024-09-12T02:18:34.349 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-12T02:18:32.459472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667795,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1272,"num_read_kb":767,"num_write":449,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360456,"kb_used_data":11208,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581368,"statfs":{"total":1151940427776,"available":1151571320832,"internally_reserved":0,"allocated":11476992,"data_stored":7256061,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8236,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":62,"num_write":16,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002781"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451322+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451322+0000","last_peered":"2024-09-12T02:18:06.451322+0000","last_clean":"2024-09-12T02:18:06.451322+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451322+0000","last_undegraded":"2024-09-12T02:18:06.451322+0000","last_fullsized":"2024-09-12T02:18:06.451322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.893300+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:29.893300+0000","last_peered":"2024-09-12T02:18:29.893300+0000","last_clean":"2024-09-12T02:18:29.893300+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:29.893300+0000","last_undegraded":"2024-09-12T02:18:29.893300+0000","last_fullsized":"2024-09-12T02:18:29.893300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":246,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891273+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:28.891273+0000","last_peered":"2024-09-12T02:18:28.891273+0000","last_clean":"2024-09-12T02:18:28.891273+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:28.891273+0000","last_undegraded":"2024-09-12T02:18:28.891273+0000","last_fullsized":"2024-09-12T02:18:28.891273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'40","reported_seq":294,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891246+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:28.891246+0000","last_peered":"2024-09-12T02:18:28.891246+0000","last_clean":"2024-09-12T02:18:28.891246+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:28.891246+0000","last_undegraded":"2024-09-12T02:18:28.891246+0000","last_fullsized":"2024-09-12T02:18:28.891246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69536,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":184,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:25.888008+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:25.888008+0000","last_peered":"2024-09-12T02:18:25.888008+0000","last_clean":"2024-09-12T02:18:25.888008+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:25.888008+0000","last_undegraded":"2024-09-12T02:18:25.888008+0000","last_fullsized":"2024-09-12T02:18:25.888008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.893211+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:29.893211+0000","last_peered":"2024-09-12T02:18:29.893211+0000","last_clean":"2024-09-12T02:18:29.893211+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:29.893211+0000","last_undegraded":"2024-09-12T02:18:29.893211+0000","last_fullsized":"2024-09-12T02:18:29.893211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.749325+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:39.749325+0000","last_peered":"2024-09-12T02:16:39.749325+0000","last_clean":"2024-09-12T02:16:39.749325+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:39.749325+0000","last_undegraded":"2024-09-12T02:16:39.749325+0000","last_fullsized":"2024-09-12T02:16:39.749325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747039+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:39.747039+0000","last_peered":"2024-09-12T02:16:39.747039+0000","last_clean":"2024-09-12T02:16:39.747039+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:39.747039+0000","last_undegraded":"2024-09-12T02:16:39.747039+0000","last_fullsized":"2024-09-12T02:16:39.747039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:26.888577+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:26.888577+0000","last_peered":"2024-09-12T02:18:26.888577+0000","last_clean":"2024-09-12T02:18:26.888577+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:26.888577+0000","last_undegraded":"2024-09-12T02:18:26.888577+0000","last_fullsized":"2024-09-12T02:18:26.888577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447635+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447635+0000","last_peered":"2024-09-12T02:18:06.447635+0000","last_clean":"2024-09-12T02:18:06.447635+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447635+0000","last_undegraded":"2024-09-12T02:18:06.447635+0000","last_fullsized":"2024-09-12T02:18:06.447635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:26.888498+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:26.888498+0000","last_peered":"2024-09-12T02:18:26.888498+0000","last_clean":"2024-09-12T02:18:26.888498+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:26.888498+0000","last_undegraded":"2024-09-12T02:18:26.888498+0000","last_fullsized":"2024-09-12T02:18:26.888498+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738039+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.738039+0000","last_peered":"2024-09-12T02:16:39.738039+0000","last_clean":"2024-09-12T02:16:39.738039+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.738039+0000","last_undegraded":"2024-09-12T02:16:39.738039+0000","last_fullsized":"2024-09-12T02:16:39.738039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'17","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.137207+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:18:21.137207+0000","last_peered":"2024-09-12T02:18:21.137207+0000","last_clean":"2024-09-12T02:18:21.137207+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:18:21.137207+0000","last_undegraded":"2024-09-12T02:18:21.137207+0000","last_fullsized":"2024-09-12T02:18:21.137207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157535+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.157535+0000","last_peered":"2024-09-12T02:16:38.157535+0000","last_clean":"2024-09-12T02:16:38.157535+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.157535+0000","last_undegraded":"2024-09-12T02:16:38.157535+0000","last_fullsized":"2024-09-12T02:16:38.157535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464122+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.464122+0000","last_peered":"2024-09-12T02:18:06.464122+0000","last_clean":"2024-09-12T02:18:06.464122+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.464122+0000","last_undegraded":"2024-09-12T02:18:06.464122+0000","last_fullsized":"2024-09-12T02:18:06.464122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.891913+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:29.891913+0000","last_peered":"2024-09-12T02:18:29.891913+0000","last_clean":"2024-09-12T02:18:29.891913+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:29.891913+0000","last_undegraded":"2024-09-12T02:18:29.891913+0000","last_fullsized":"2024-09-12T02:18:29.891913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1146,"num_read_kb":658,"num_write":266,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":231579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1100181,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444116,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":509231,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":509545,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:34.354 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:18:34.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:18:34.611 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:34.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:34 smithi003 ceph-mon[31476]: pgmap v341: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T02:18:34.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:34 smithi026 ceph-mon[35496]: pgmap v341: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T02:18:34.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:34 smithi092 ceph-mon[36089]: pgmap v341: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T02:18:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 -- 172.21.15.3:0/3720283490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 msgr2=0x7f37481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 --2- 172.21.15.3:0/3720283490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f37481021c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3744009900 tx=0x7f374402a410 comp rx=0 tx=0).stop 2024-09-12T02:18:35.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 -- 172.21.15.3:0/3720283490 shutdown_connections 2024-09-12T02:18:35.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 --2- 172.21.15.3:0/3720283490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 0x7f374810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 --2- 172.21.15.3:0/3720283490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 0x7f37481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 --2- 172.21.15.3:0/3720283490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f37481021c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 -- 172.21.15.3:0/3720283490 >> 172.21.15.3:0/3720283490 conn(0x7f37480fd8c0 msgr2=0x7f37480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:35.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 -- 172.21.15.3:0/3720283490 shutdown_connections 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.081+0000 7f374fba2640 1 -- 172.21.15.3:0/3720283490 wait complete. 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 Processor -- start 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 -- start start 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 0x7f374819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:35.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 0x7f374819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3748111f90 con 0x7f3748101de0 2024-09-12T02:18:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3748111e10 con 0x7f3748102790 2024-09-12T02:18:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374fba2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3748112110 con 0x7f3748106f10 2024-09-12T02:18:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:35.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57338/0 (socket says 172.21.15.3:57338) 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 -- 172.21.15.3:0/1780425909 learned_addr learned my addr 172.21.15.3:0/1780425909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374e13f640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 0x7f374819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d13d640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 0x7f374819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 -- 172.21.15.3:0/1780425909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 msgr2=0x7f374819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 0x7f374819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 -- 172.21.15.3:0/1780425909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 msgr2=0x7f374819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 0x7f374819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 -- 172.21.15.3:0/1780425909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374819f690 con 0x7f3748101de0 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.082+0000 7f374d93e640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3744005ab0 tx=0x7f37440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.083+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374403f030 con 0x7f3748101de0 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.083+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3744003ce0 con 0x7f3748101de0 2024-09-12T02:18:35.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.083+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374819f920 con 0x7f3748101de0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.083+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744035070 con 0x7f3748101de0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.083+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481a7260 con 0x7f3748101de0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.084+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37481021c0 con 0x7f3748101de0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.085+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3744042030 con 0x7f3748101de0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.086+0000 7f373effd640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 0x7f37340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.086+0000 7f374d13d640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 0x7f37340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:35.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.086+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f37440efce0 con 0x7f3748101de0 2024-09-12T02:18:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.086+0000 7f374d13d640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 0x7f37340a5150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f374810eee0 tx=0x7f3738009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:35.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.089+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37440bca70 con 0x7f3748101de0 2024-09-12T02:18:35.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.263+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 --> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f374810de00 con 0x7f37340a2c90 2024-09-12T02:18:35.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.269+0000 7f373effd640 1 -- 172.21.15.3:0/1780425909 <== mgr.14150 v2:172.21.15.3:6800/2546806763 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259113 (secure 0 0 0) 0x7f374810de00 con 0x7f37340a2c90 2024-09-12T02:18:35.274 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:35.276 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 msgr2=0x7f37340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 0x7f37340a5150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f374810eee0 tx=0x7f3738009110 comp rx=0 tx=0).stop 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 msgr2=0x7f374819a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3744005ab0 tx=0x7f37440042f0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 shutdown_connections 2024-09-12T02:18:35.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f37340a2c90 0x7f37340a5150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3748106f10 0x7f374819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3748102790 0x7f374819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 --2- 172.21.15.3:0/1780425909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3748101de0 0x7f374819a780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 >> 172.21.15.3:0/1780425909 conn(0x7f37480fd8c0 msgr2=0x7f37480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:35.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 shutdown_connections 2024-09-12T02:18:35.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:35.272+0000 7f374fba2640 1 -- 172.21.15.3:0/1780425909 wait complete. 2024-09-12T02:18:35.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:35 smithi003 ceph-mon[31476]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:35.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:35 smithi026 ceph-mon[35496]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:35.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:35 smithi092 ceph-mon[36089]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:35.972 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-12T02:18:34.459865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667795,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1286,"num_read_kb":771,"num_write":449,"num_write_kb":3098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360456,"kb_used_data":11208,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581368,"statfs":{"total":1151940427776,"available":1151571320832,"internally_reserved":0,"allocated":11476992,"data_stored":7256061,"data_compressed":178497,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8236,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":66,"num_write":16,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460268+0000","last_change":"2024-09-12T02:16:29.127834+0000","last_active":"2024-09-12T02:18:06.460268+0000","last_peered":"2024-09-12T02:18:06.460268+0000","last_clean":"2024-09-12T02:18:06.460268+0000","last_became_active":"2024-09-12T02:16:29.127458+0000","last_became_peered":"2024-09-12T02:16:29.127458+0000","last_unstale":"2024-09-12T02:18:06.460268+0000","last_undegraded":"2024-09-12T02:18:06.460268+0000","last_fullsized":"2024-09-12T02:18:06.460268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:28:02.090809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447038+0000","last_change":"2024-09-12T02:16:29.133137+0000","last_active":"2024-09-12T02:18:06.447038+0000","last_peered":"2024-09-12T02:18:06.447038+0000","last_clean":"2024-09-12T02:18:06.447038+0000","last_became_active":"2024-09-12T02:16:29.132797+0000","last_became_peered":"2024-09-12T02:16:29.132797+0000","last_unstale":"2024-09-12T02:18:06.447038+0000","last_undegraded":"2024-09-12T02:18:06.447038+0000","last_fullsized":"2024-09-12T02:18:06.447038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:12:28.041410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455727+0000","last_change":"2024-09-12T02:16:29.128992+0000","last_active":"2024-09-12T02:18:06.455727+0000","last_peered":"2024-09-12T02:18:06.455727+0000","last_clean":"2024-09-12T02:18:06.455727+0000","last_became_active":"2024-09-12T02:16:29.127468+0000","last_became_peered":"2024-09-12T02:16:29.127468+0000","last_unstale":"2024-09-12T02:18:06.455727+0000","last_undegraded":"2024-09-12T02:18:06.455727+0000","last_fullsized":"2024-09-12T02:18:06.455727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:35:07.220227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451438+0000","last_change":"2024-09-12T02:16:29.135330+0000","last_active":"2024-09-12T02:18:06.451438+0000","last_peered":"2024-09-12T02:18:06.451438+0000","last_clean":"2024-09-12T02:18:06.451438+0000","last_became_active":"2024-09-12T02:16:29.134994+0000","last_became_peered":"2024-09-12T02:16:29.134994+0000","last_unstale":"2024-09-12T02:18:06.451438+0000","last_undegraded":"2024-09-12T02:18:06.451438+0000","last_fullsized":"2024-09-12T02:18:06.451438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:54.092955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.125256+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.124923+0000","last_became_peered":"2024-09-12T02:16:29.124923+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:05:12.943812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438171+0000","last_change":"2024-09-12T02:16:29.131118+0000","last_active":"2024-09-12T02:18:06.438171+0000","last_peered":"2024-09-12T02:18:06.438171+0000","last_clean":"2024-09-12T02:18:06.438171+0000","last_became_active":"2024-09-12T02:16:29.130729+0000","last_became_peered":"2024-09-12T02:16:29.130729+0000","last_unstale":"2024-09-12T02:18:06.438171+0000","last_undegraded":"2024-09-12T02:18:06.438171+0000","last_fullsized":"2024-09-12T02:18:06.438171+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:37.477598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458250+0000","last_change":"2024-09-12T02:16:38.160088+0000","last_active":"2024-09-12T02:18:06.458250+0000","last_peered":"2024-09-12T02:18:06.458250+0000","last_clean":"2024-09-12T02:18:06.458250+0000","last_became_active":"2024-09-12T02:16:38.159317+0000","last_became_peered":"2024-09-12T02:16:38.159317+0000","last_unstale":"2024-09-12T02:18:06.458250+0000","last_undegraded":"2024-09-12T02:18:06.458250+0000","last_fullsized":"2024-09-12T02:18:06.458250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:09:03.049360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464364+0000","last_change":"2024-09-12T02:16:29.125877+0000","last_active":"2024-09-12T02:18:06.464364+0000","last_peered":"2024-09-12T02:18:06.464364+0000","last_clean":"2024-09-12T02:18:06.464364+0000","last_became_active":"2024-09-12T02:16:29.124960+0000","last_became_peered":"2024-09-12T02:16:29.124960+0000","last_unstale":"2024-09-12T02:18:06.464364+0000","last_undegraded":"2024-09-12T02:18:06.464364+0000","last_fullsized":"2024-09-12T02:18:06.464364+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:09.086548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460008+0000","last_change":"2024-09-12T02:16:29.126326+0000","last_active":"2024-09-12T02:18:06.460008+0000","last_peered":"2024-09-12T02:18:06.460008+0000","last_clean":"2024-09-12T02:18:06.460008+0000","last_became_active":"2024-09-12T02:16:29.126004+0000","last_became_peered":"2024-09-12T02:16:29.126004+0000","last_unstale":"2024-09-12T02:18:06.460008+0000","last_undegraded":"2024-09-12T02:18:06.460008+0000","last_fullsized":"2024-09-12T02:18:06.460008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:39.720646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440837+0000","last_change":"2024-09-12T02:16:29.133841+0000","last_active":"2024-09-12T02:18:06.440837+0000","last_peered":"2024-09-12T02:18:06.440837+0000","last_clean":"2024-09-12T02:18:06.440837+0000","last_became_active":"2024-09-12T02:16:29.133442+0000","last_became_peered":"2024-09-12T02:16:29.133442+0000","last_unstale":"2024-09-12T02:18:06.440837+0000","last_undegraded":"2024-09-12T02:18:06.440837+0000","last_fullsized":"2024-09-12T02:18:06.440837+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:30.879486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455006+0000","last_change":"2024-09-12T02:16:29.129044+0000","last_active":"2024-09-12T02:18:06.455006+0000","last_peered":"2024-09-12T02:18:06.455006+0000","last_clean":"2024-09-12T02:18:06.455006+0000","last_became_active":"2024-09-12T02:16:29.125783+0000","last_became_peered":"2024-09-12T02:16:29.125783+0000","last_unstale":"2024-09-12T02:18:06.455006+0000","last_undegraded":"2024-09-12T02:18:06.455006+0000","last_fullsized":"2024-09-12T02:18:06.455006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:08:42.300101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450985+0000","last_change":"2024-09-12T02:16:29.132646+0000","last_active":"2024-09-12T02:18:06.450985+0000","last_peered":"2024-09-12T02:18:06.450985+0000","last_clean":"2024-09-12T02:18:06.450985+0000","last_became_active":"2024-09-12T02:16:29.131985+0000","last_became_peered":"2024-09-12T02:16:29.131985+0000","last_unstale":"2024-09-12T02:18:06.450985+0000","last_undegraded":"2024-09-12T02:18:06.450985+0000","last_fullsized":"2024-09-12T02:18:06.450985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:36.987520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.457977+0000","last_change":"2024-09-12T02:16:29.121635+0000","last_active":"2024-09-12T02:18:06.457977+0000","last_peered":"2024-09-12T02:18:06.457977+0000","last_clean":"2024-09-12T02:18:06.457977+0000","last_became_active":"2024-09-12T02:16:29.121342+0000","last_became_peered":"2024-09-12T02:16:29.121342+0000","last_unstale":"2024-09-12T02:18:06.457977+0000","last_undegraded":"2024-09-12T02:18:06.457977+0000","last_fullsized":"2024-09-12T02:18:06.457977+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:11.223881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439879+0000","last_change":"2024-09-12T02:16:29.133311+0000","last_active":"2024-09-12T02:18:06.439879+0000","last_peered":"2024-09-12T02:18:06.439879+0000","last_clean":"2024-09-12T02:18:06.439879+0000","last_became_active":"2024-09-12T02:16:29.132985+0000","last_became_peered":"2024-09-12T02:16:29.132985+0000","last_unstale":"2024-09-12T02:18:06.439879+0000","last_undegraded":"2024-09-12T02:18:06.439879+0000","last_fullsized":"2024-09-12T02:18:06.439879+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:40.070777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455452+0000","last_change":"2024-09-12T02:16:29.128172+0000","last_active":"2024-09-12T02:18:06.455452+0000","last_peered":"2024-09-12T02:18:06.455452+0000","last_clean":"2024-09-12T02:18:06.455452+0000","last_became_active":"2024-09-12T02:16:29.127452+0000","last_became_peered":"2024-09-12T02:16:29.127452+0000","last_unstale":"2024-09-12T02:18:06.455452+0000","last_undegraded":"2024-09-12T02:18:06.455452+0000","last_fullsized":"2024-09-12T02:18:06.455452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:23:14.190482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459062+0000","last_change":"2024-09-12T02:16:29.137940+0000","last_active":"2024-09-12T02:18:06.459062+0000","last_peered":"2024-09-12T02:18:06.459062+0000","last_clean":"2024-09-12T02:18:06.459062+0000","last_became_active":"2024-09-12T02:16:29.137608+0000","last_became_peered":"2024-09-12T02:16:29.137608+0000","last_unstale":"2024-09-12T02:18:06.459062+0000","last_undegraded":"2024-09-12T02:18:06.459062+0000","last_fullsized":"2024-09-12T02:18:06.459062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.691841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460443+0000","last_change":"2024-09-12T02:16:29.127841+0000","last_active":"2024-09-12T02:18:06.460443+0000","last_peered":"2024-09-12T02:18:06.460443+0000","last_clean":"2024-09-12T02:18:06.460443+0000","last_became_active":"2024-09-12T02:16:29.127206+0000","last_became_peered":"2024-09-12T02:16:29.127206+0000","last_unstale":"2024-09-12T02:18:06.460443+0000","last_undegraded":"2024-09-12T02:18:06.460443+0000","last_fullsized":"2024-09-12T02:18:06.460443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:47.693630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.439258+0000","last_change":"2024-09-12T02:16:29.123224+0000","last_active":"2024-09-12T02:18:06.439258+0000","last_peered":"2024-09-12T02:18:06.439258+0000","last_clean":"2024-09-12T02:18:06.439258+0000","last_became_active":"2024-09-12T02:16:29.122903+0000","last_became_peered":"2024-09-12T02:16:29.122903+0000","last_unstale":"2024-09-12T02:18:06.439258+0000","last_undegraded":"2024-09-12T02:18:06.439258+0000","last_fullsized":"2024-09-12T02:18:06.439258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:56:25.951352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448598+0000","last_change":"2024-09-12T02:16:29.126458+0000","last_active":"2024-09-12T02:18:06.448598+0000","last_peered":"2024-09-12T02:18:06.448598+0000","last_clean":"2024-09-12T02:18:06.448598+0000","last_became_active":"2024-09-12T02:16:29.126151+0000","last_became_peered":"2024-09-12T02:16:29.126151+0000","last_unstale":"2024-09-12T02:18:06.448598+0000","last_undegraded":"2024-09-12T02:18:06.448598+0000","last_fullsized":"2024-09-12T02:18:06.448598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:06.856982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452725+0000","last_change":"2024-09-12T02:16:29.128887+0000","last_active":"2024-09-12T02:18:06.452725+0000","last_peered":"2024-09-12T02:18:06.452725+0000","last_clean":"2024-09-12T02:18:06.452725+0000","last_became_active":"2024-09-12T02:16:29.128588+0000","last_became_peered":"2024-09-12T02:16:29.128588+0000","last_unstale":"2024-09-12T02:18:06.452725+0000","last_undegraded":"2024-09-12T02:18:06.452725+0000","last_fullsized":"2024-09-12T02:18:06.452725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:43:45.436400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463573+0000","last_change":"2024-09-12T02:16:29.124435+0000","last_active":"2024-09-12T02:18:06.463573+0000","last_peered":"2024-09-12T02:18:06.463573+0000","last_clean":"2024-09-12T02:18:06.463573+0000","last_became_active":"2024-09-12T02:16:29.124137+0000","last_became_peered":"2024-09-12T02:16:29.124137+0000","last_unstale":"2024-09-12T02:18:06.463573+0000","last_undegraded":"2024-09-12T02:18:06.463573+0000","last_fullsized":"2024-09-12T02:18:06.463573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:07:25.450397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462703+0000","last_change":"2024-09-12T02:16:29.128754+0000","last_active":"2024-09-12T02:18:06.462703+0000","last_peered":"2024-09-12T02:18:06.462703+0000","last_clean":"2024-09-12T02:18:06.462703+0000","last_became_active":"2024-09-12T02:16:29.128434+0000","last_became_peered":"2024-09-12T02:16:29.128434+0000","last_unstale":"2024-09-12T02:18:06.462703+0000","last_undegraded":"2024-09-12T02:18:06.462703+0000","last_fullsized":"2024-09-12T02:18:06.462703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:05.900113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462264+0000","last_change":"2024-09-12T02:16:29.130677+0000","last_active":"2024-09-12T02:18:06.462264+0000","last_peered":"2024-09-12T02:18:06.462264+0000","last_clean":"2024-09-12T02:18:06.462264+0000","last_became_active":"2024-09-12T02:16:29.128617+0000","last_became_peered":"2024-09-12T02:16:29.128617+0000","last_unstale":"2024-09-12T02:18:06.462264+0000","last_undegraded":"2024-09-12T02:18:06.462264+0000","last_fullsized":"2024-09-12T02:18:06.462264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:09.054722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.445341+0000","last_change":"2024-09-12T02:16:29.120172+0000","last_active":"2024-09-12T02:18:06.445341+0000","last_peered":"2024-09-12T02:18:06.445341+0000","last_clean":"2024-09-12T02:18:06.445341+0000","last_became_active":"2024-09-12T02:16:29.119864+0000","last_became_peered":"2024-09-12T02:16:29.119864+0000","last_unstale":"2024-09-12T02:18:06.445341+0000","last_undegraded":"2024-09-12T02:18:06.445341+0000","last_fullsized":"2024-09-12T02:18:06.445341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:30:11.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453599+0000","last_change":"2024-09-12T02:16:29.132157+0000","last_active":"2024-09-12T02:18:06.453599+0000","last_peered":"2024-09-12T02:18:06.453599+0000","last_clean":"2024-09-12T02:18:06.453599+0000","last_became_active":"2024-09-12T02:16:29.131847+0000","last_became_peered":"2024-09-12T02:16:29.131847+0000","last_unstale":"2024-09-12T02:18:06.453599+0000","last_undegraded":"2024-09-12T02:18:06.453599+0000","last_fullsized":"2024-09-12T02:18:06.453599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:02.087783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.450278+0000","last_change":"2024-09-12T02:16:29.127282+0000","last_active":"2024-09-12T02:18:06.450278+0000","last_peered":"2024-09-12T02:18:06.450278+0000","last_clean":"2024-09-12T02:18:06.450278+0000","last_became_active":"2024-09-12T02:16:29.126758+0000","last_became_peered":"2024-09-12T02:16:29.126758+0000","last_unstale":"2024-09-12T02:18:06.450278+0000","last_undegraded":"2024-09-12T02:18:06.450278+0000","last_fullsized":"2024-09-12T02:18:06.450278+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:10.024054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440054+0000","last_change":"2024-09-12T02:16:29.118251+0000","last_active":"2024-09-12T02:18:06.440054+0000","last_peered":"2024-09-12T02:18:06.440054+0000","last_clean":"2024-09-12T02:18:06.440054+0000","last_became_active":"2024-09-12T02:16:29.117419+0000","last_became_peered":"2024-09-12T02:16:29.117419+0000","last_unstale":"2024-09-12T02:18:06.440054+0000","last_undegraded":"2024-09-12T02:18:06.440054+0000","last_fullsized":"2024-09-12T02:18:06.440054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:06.361137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460424+0000","last_change":"2024-09-12T02:16:29.129192+0000","last_active":"2024-09-12T02:18:06.460424+0000","last_peered":"2024-09-12T02:18:06.460424+0000","last_clean":"2024-09-12T02:18:06.460424+0000","last_became_active":"2024-09-12T02:16:29.128817+0000","last_became_peered":"2024-09-12T02:16:29.128817+0000","last_unstale":"2024-09-12T02:18:06.460424+0000","last_undegraded":"2024-09-12T02:18:06.460424+0000","last_fullsized":"2024-09-12T02:18:06.460424+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:44.093644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455026+0000","last_change":"2024-09-12T02:16:38.159557+0000","last_active":"2024-09-12T02:18:06.455026+0000","last_peered":"2024-09-12T02:18:06.455026+0000","last_clean":"2024-09-12T02:18:06.455026+0000","last_became_active":"2024-09-12T02:16:38.159221+0000","last_became_peered":"2024-09-12T02:16:38.159221+0000","last_unstale":"2024-09-12T02:18:06.455026+0000","last_undegraded":"2024-09-12T02:18:06.455026+0000","last_fullsized":"2024-09-12T02:18:06.455026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:56.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451474+0000","last_change":"2024-09-12T02:16:29.132870+0000","last_active":"2024-09-12T02:18:06.451474+0000","last_peered":"2024-09-12T02:18:06.451474+0000","last_clean":"2024-09-12T02:18:06.451474+0000","last_became_active":"2024-09-12T02:16:29.132524+0000","last_became_peered":"2024-09-12T02:16:29.132524+0000","last_unstale":"2024-09-12T02:18:06.451474+0000","last_undegraded":"2024-09-12T02:18:06.451474+0000","last_fullsized":"2024-09-12T02:18:06.451474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:49.749032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455614+0000","last_change":"2024-09-12T02:16:38.159777+0000","last_active":"2024-09-12T02:18:06.455614+0000","last_peered":"2024-09-12T02:18:06.455614+0000","last_clean":"2024-09-12T02:18:06.455614+0000","last_became_active":"2024-09-12T02:16:38.159381+0000","last_became_peered":"2024-09-12T02:16:38.159381+0000","last_unstale":"2024-09-12T02:18:06.455614+0000","last_undegraded":"2024-09-12T02:18:06.455614+0000","last_fullsized":"2024-09-12T02:18:06.455614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:54.803292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.453156+0000","last_change":"2024-09-12T02:16:29.121568+0000","last_active":"2024-09-12T02:18:06.453156+0000","last_peered":"2024-09-12T02:18:06.453156+0000","last_clean":"2024-09-12T02:18:06.453156+0000","last_became_active":"2024-09-12T02:16:29.121095+0000","last_became_peered":"2024-09-12T02:16:29.121095+0000","last_unstale":"2024-09-12T02:18:06.453156+0000","last_undegraded":"2024-09-12T02:18:06.453156+0000","last_fullsized":"2024-09-12T02:18:06.453156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:26:53.957409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440190+0000","last_change":"2024-09-12T02:16:29.122685+0000","last_active":"2024-09-12T02:18:06.440190+0000","last_peered":"2024-09-12T02:18:06.440190+0000","last_clean":"2024-09-12T02:18:06.440190+0000","last_became_active":"2024-09-12T02:16:29.122358+0000","last_became_peered":"2024-09-12T02:16:29.122358+0000","last_unstale":"2024-09-12T02:18:06.440190+0000","last_undegraded":"2024-09-12T02:18:06.440190+0000","last_fullsized":"2024-09-12T02:18:06.440190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:52:12.151699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.893300+0000","last_change":"2024-09-12T02:16:27.089628+0000","last_active":"2024-09-12T02:18:29.893300+0000","last_peered":"2024-09-12T02:18:29.893300+0000","last_clean":"2024-09-12T02:18:29.893300+0000","last_became_active":"2024-09-12T02:16:27.089275+0000","last_became_peered":"2024-09-12T02:16:27.089275+0000","last_unstale":"2024-09-12T02:18:29.893300+0000","last_undegraded":"2024-09-12T02:18:29.893300+0000","last_fullsized":"2024-09-12T02:18:29.893300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:17.171610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.448743+0000","last_change":"2024-09-12T02:16:29.130403+0000","last_active":"2024-09-12T02:18:06.448743+0000","last_peered":"2024-09-12T02:18:06.448743+0000","last_clean":"2024-09-12T02:18:06.448743+0000","last_became_active":"2024-09-12T02:16:29.129849+0000","last_became_peered":"2024-09-12T02:16:29.129849+0000","last_unstale":"2024-09-12T02:18:06.448743+0000","last_undegraded":"2024-09-12T02:18:06.448743+0000","last_fullsized":"2024-09-12T02:18:06.448743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:40:49.212558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.171457+0000","last_change":"2024-09-12T02:16:27.089229+0000","last_active":"2024-09-12T02:16:39.171457+0000","last_peered":"2024-09-12T02:16:39.171457+0000","last_clean":"2024-09-12T02:16:39.171457+0000","last_became_active":"2024-09-12T02:16:27.088797+0000","last_became_peered":"2024-09-12T02:16:27.088797+0000","last_unstale":"2024-09-12T02:16:39.171457+0000","last_undegraded":"2024-09-12T02:16:39.171457+0000","last_fullsized":"2024-09-12T02:16:39.171457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:12.084356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.162154+0000","last_change":"2024-09-12T02:16:27.087778+0000","last_active":"2024-09-12T02:16:38.162154+0000","last_peered":"2024-09-12T02:16:38.162154+0000","last_clean":"2024-09-12T02:16:38.162154+0000","last_became_active":"2024-09-12T02:16:27.087410+0000","last_became_peered":"2024-09-12T02:16:27.087410+0000","last_unstale":"2024-09-12T02:16:38.162154+0000","last_undegraded":"2024-09-12T02:16:38.162154+0000","last_fullsized":"2024-09-12T02:16:38.162154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:29:31.545876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.462415+0000","last_change":"2024-09-12T02:16:29.118621+0000","last_active":"2024-09-12T02:18:06.462415+0000","last_peered":"2024-09-12T02:18:06.462415+0000","last_clean":"2024-09-12T02:18:06.462415+0000","last_became_active":"2024-09-12T02:16:29.118302+0000","last_became_peered":"2024-09-12T02:16:29.118302+0000","last_unstale":"2024-09-12T02:18:06.462415+0000","last_undegraded":"2024-09-12T02:18:06.462415+0000","last_fullsized":"2024-09-12T02:18:06.462415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:52.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.744952+0000","last_change":"2024-09-12T02:16:27.088972+0000","last_active":"2024-09-12T02:16:39.744952+0000","last_peered":"2024-09-12T02:16:39.744952+0000","last_clean":"2024-09-12T02:16:39.744952+0000","last_became_active":"2024-09-12T02:16:27.088651+0000","last_became_peered":"2024-09-12T02:16:27.088651+0000","last_unstale":"2024-09-12T02:16:39.744952+0000","last_undegraded":"2024-09-12T02:16:39.744952+0000","last_fullsized":"2024-09-12T02:16:39.744952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:28:15.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459031+0000","last_change":"2024-09-12T02:16:29.122736+0000","last_active":"2024-09-12T02:18:06.459031+0000","last_peered":"2024-09-12T02:18:06.459031+0000","last_clean":"2024-09-12T02:18:06.459031+0000","last_became_active":"2024-09-12T02:16:29.122353+0000","last_became_peered":"2024-09-12T02:16:29.122353+0000","last_unstale":"2024-09-12T02:18:06.459031+0000","last_undegraded":"2024-09-12T02:18:06.459031+0000","last_fullsized":"2024-09-12T02:18:06.459031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:54.875082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":246,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891273+0000","last_change":"2024-09-12T02:16:27.090974+0000","last_active":"2024-09-12T02:18:28.891273+0000","last_peered":"2024-09-12T02:18:28.891273+0000","last_clean":"2024-09-12T02:18:28.891273+0000","last_became_active":"2024-09-12T02:16:27.090546+0000","last_became_peered":"2024-09-12T02:16:27.090546+0000","last_unstale":"2024-09-12T02:18:28.891273+0000","last_undegraded":"2024-09-12T02:18:28.891273+0000","last_fullsized":"2024-09-12T02:18:28.891273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:26.728809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":220,"num_read_kb":220,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460695+0000","last_change":"2024-09-12T02:16:29.130305+0000","last_active":"2024-09-12T02:18:06.460695+0000","last_peered":"2024-09-12T02:18:06.460695+0000","last_clean":"2024-09-12T02:18:06.460695+0000","last_became_active":"2024-09-12T02:16:29.129917+0000","last_became_peered":"2024-09-12T02:16:29.129917+0000","last_unstale":"2024-09-12T02:18:06.460695+0000","last_undegraded":"2024-09-12T02:18:06.460695+0000","last_fullsized":"2024-09-12T02:18:06.460695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:17:32.245400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747292+0000","last_change":"2024-09-12T02:16:27.091606+0000","last_active":"2024-09-12T02:16:39.747292+0000","last_peered":"2024-09-12T02:16:39.747292+0000","last_clean":"2024-09-12T02:16:39.747292+0000","last_became_active":"2024-09-12T02:16:27.091122+0000","last_became_peered":"2024-09-12T02:16:27.091122+0000","last_unstale":"2024-09-12T02:16:39.747292+0000","last_undegraded":"2024-09-12T02:16:39.747292+0000","last_fullsized":"2024-09-12T02:16:39.747292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:56.963998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438985+0000","last_change":"2024-09-12T02:16:29.131014+0000","last_active":"2024-09-12T02:18:06.438985+0000","last_peered":"2024-09-12T02:18:06.438985+0000","last_clean":"2024-09-12T02:18:06.438985+0000","last_became_active":"2024-09-12T02:16:29.130465+0000","last_became_peered":"2024-09-12T02:16:29.130465+0000","last_unstale":"2024-09-12T02:18:06.438985+0000","last_undegraded":"2024-09-12T02:18:06.438985+0000","last_fullsized":"2024-09-12T02:18:06.438985+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:04:00.518874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170420+0000","last_change":"2024-09-12T02:16:27.117068+0000","last_active":"2024-09-12T02:16:39.170420+0000","last_peered":"2024-09-12T02:16:39.170420+0000","last_clean":"2024-09-12T02:16:39.170420+0000","last_became_active":"2024-09-12T02:16:27.116624+0000","last_became_peered":"2024-09-12T02:16:27.116624+0000","last_unstale":"2024-09-12T02:16:39.170420+0000","last_undegraded":"2024-09-12T02:16:39.170420+0000","last_fullsized":"2024-09-12T02:16:39.170420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:38.692083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460963+0000","last_change":"2024-09-12T02:16:29.134379+0000","last_active":"2024-09-12T02:18:06.460963+0000","last_peered":"2024-09-12T02:18:06.460963+0000","last_clean":"2024-09-12T02:18:06.460963+0000","last_became_active":"2024-09-12T02:16:29.134027+0000","last_became_peered":"2024-09-12T02:16:29.134027+0000","last_unstale":"2024-09-12T02:18:06.460963+0000","last_undegraded":"2024-09-12T02:18:06.460963+0000","last_fullsized":"2024-09-12T02:18:06.460963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:29:36.783341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'40","reported_seq":294,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:28.891246+0000","last_change":"2024-09-12T02:16:27.090702+0000","last_active":"2024-09-12T02:18:28.891246+0000","last_peered":"2024-09-12T02:18:28.891246+0000","last_clean":"2024-09-12T02:18:28.891246+0000","last_became_active":"2024-09-12T02:16:27.090386+0000","last_became_peered":"2024-09-12T02:16:27.090386+0000","last_unstale":"2024-09-12T02:18:28.891246+0000","last_undegraded":"2024-09-12T02:18:28.891246+0000","last_fullsized":"2024-09-12T02:18:28.891246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:05:29.359201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69536,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":232,"num_read_kb":184,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455061+0000","last_change":"2024-09-12T02:16:29.124956+0000","last_active":"2024-09-12T02:18:06.455061+0000","last_peered":"2024-09-12T02:18:06.455061+0000","last_clean":"2024-09-12T02:18:06.455061+0000","last_became_active":"2024-09-12T02:16:29.124192+0000","last_became_peered":"2024-09-12T02:16:29.124192+0000","last_unstale":"2024-09-12T02:18:06.455061+0000","last_undegraded":"2024-09-12T02:18:06.455061+0000","last_fullsized":"2024-09-12T02:18:06.455061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:55.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748642+0000","last_change":"2024-09-12T02:16:27.089715+0000","last_active":"2024-09-12T02:16:39.748642+0000","last_peered":"2024-09-12T02:16:39.748642+0000","last_clean":"2024-09-12T02:16:39.748642+0000","last_became_active":"2024-09-12T02:16:27.089390+0000","last_became_peered":"2024-09-12T02:16:27.089390+0000","last_unstale":"2024-09-12T02:16:39.748642+0000","last_undegraded":"2024-09-12T02:16:39.748642+0000","last_fullsized":"2024-09-12T02:16:39.748642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:18:55.976148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.463979+0000","last_change":"2024-09-12T02:16:29.123621+0000","last_active":"2024-09-12T02:18:06.463979+0000","last_peered":"2024-09-12T02:18:06.463979+0000","last_clean":"2024-09-12T02:18:06.463979+0000","last_became_active":"2024-09-12T02:16:29.123289+0000","last_became_peered":"2024-09-12T02:16:29.123289+0000","last_unstale":"2024-09-12T02:18:06.463979+0000","last_undegraded":"2024-09-12T02:18:06.463979+0000","last_fullsized":"2024-09-12T02:18:06.463979+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:30.000128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747482+0000","last_change":"2024-09-12T02:16:27.088238+0000","last_active":"2024-09-12T02:16:39.747482+0000","last_peered":"2024-09-12T02:16:39.747482+0000","last_clean":"2024-09-12T02:16:39.747482+0000","last_became_active":"2024-09-12T02:16:27.087856+0000","last_became_peered":"2024-09-12T02:16:27.087856+0000","last_unstale":"2024-09-12T02:16:39.747482+0000","last_undegraded":"2024-09-12T02:16:39.747482+0000","last_fullsized":"2024-09-12T02:16:39.747482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:13:52.265446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464430+0000","last_change":"2024-09-12T02:16:38.160799+0000","last_active":"2024-09-12T02:18:06.464430+0000","last_peered":"2024-09-12T02:18:06.464430+0000","last_clean":"2024-09-12T02:18:06.464430+0000","last_became_active":"2024-09-12T02:16:38.160314+0000","last_became_peered":"2024-09-12T02:16:38.160314+0000","last_unstale":"2024-09-12T02:18:06.464430+0000","last_undegraded":"2024-09-12T02:18:06.464430+0000","last_fullsized":"2024-09-12T02:18:06.464430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:56:34.161477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:30.894245+0000","last_change":"2024-09-12T02:16:27.092217+0000","last_active":"2024-09-12T02:18:30.894245+0000","last_peered":"2024-09-12T02:18:30.894245+0000","last_clean":"2024-09-12T02:18:30.894245+0000","last_became_active":"2024-09-12T02:16:27.091906+0000","last_became_peered":"2024-09-12T02:16:27.091906+0000","last_unstale":"2024-09-12T02:18:30.894245+0000","last_undegraded":"2024-09-12T02:18:30.894245+0000","last_fullsized":"2024-09-12T02:18:30.894245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:03:53.623313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456565+0000","last_change":"2024-09-12T02:16:29.122673+0000","last_active":"2024-09-12T02:18:06.456565+0000","last_peered":"2024-09-12T02:18:06.456565+0000","last_clean":"2024-09-12T02:18:06.456565+0000","last_became_active":"2024-09-12T02:16:29.122366+0000","last_became_peered":"2024-09-12T02:16:29.122366+0000","last_unstale":"2024-09-12T02:18:06.456565+0000","last_undegraded":"2024-09-12T02:18:06.456565+0000","last_fullsized":"2024-09-12T02:18:06.456565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:20:03.105761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.738845+0000","last_change":"2024-09-12T02:16:27.116823+0000","last_active":"2024-09-12T02:16:39.738845+0000","last_peered":"2024-09-12T02:16:39.738845+0000","last_clean":"2024-09-12T02:16:39.738845+0000","last_became_active":"2024-09-12T02:16:27.116205+0000","last_became_peered":"2024-09-12T02:16:27.116205+0000","last_unstale":"2024-09-12T02:16:39.738845+0000","last_undegraded":"2024-09-12T02:16:39.738845+0000","last_fullsized":"2024-09-12T02:16:39.738845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:15:31.129826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.436467+0000","last_change":"2024-09-12T02:16:38.141702+0000","last_active":"2024-09-12T02:18:06.436467+0000","last_peered":"2024-09-12T02:18:06.436467+0000","last_clean":"2024-09-12T02:18:06.436467+0000","last_became_active":"2024-09-12T02:16:38.141292+0000","last_became_peered":"2024-09-12T02:16:38.141292+0000","last_unstale":"2024-09-12T02:18:06.436467+0000","last_undegraded":"2024-09-12T02:18:06.436467+0000","last_fullsized":"2024-09-12T02:18:06.436467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:24:34.891369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.167340+0000","last_change":"2024-09-12T02:16:27.088859+0000","last_active":"2024-09-12T02:18:21.167340+0000","last_peered":"2024-09-12T02:18:21.167340+0000","last_clean":"2024-09-12T02:18:21.167340+0000","last_became_active":"2024-09-12T02:16:27.087635+0000","last_became_peered":"2024-09-12T02:16:27.087635+0000","last_unstale":"2024-09-12T02:18:21.167340+0000","last_undegraded":"2024-09-12T02:18:21.167340+0000","last_fullsized":"2024-09-12T02:18:21.167340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:04.658998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.452392+0000","last_change":"2024-09-12T02:16:29.129606+0000","last_active":"2024-09-12T02:18:06.452392+0000","last_peered":"2024-09-12T02:18:06.452392+0000","last_clean":"2024-09-12T02:18:06.452392+0000","last_became_active":"2024-09-12T02:16:29.129300+0000","last_became_peered":"2024-09-12T02:16:29.129300+0000","last_unstale":"2024-09-12T02:18:06.452392+0000","last_undegraded":"2024-09-12T02:18:06.452392+0000","last_fullsized":"2024-09-12T02:18:06.452392+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:11.153125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745517+0000","last_change":"2024-09-12T02:16:27.092930+0000","last_active":"2024-09-12T02:16:39.745517+0000","last_peered":"2024-09-12T02:16:39.745517+0000","last_clean":"2024-09-12T02:16:39.745517+0000","last_became_active":"2024-09-12T02:16:27.092528+0000","last_became_peered":"2024-09-12T02:16:27.092528+0000","last_unstale":"2024-09-12T02:16:39.745517+0000","last_undegraded":"2024-09-12T02:16:39.745517+0000","last_fullsized":"2024-09-12T02:16:39.745517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:47:32.558198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.455867+0000","last_change":"2024-09-12T02:16:29.125147+0000","last_active":"2024-09-12T02:18:06.455867+0000","last_peered":"2024-09-12T02:18:06.455867+0000","last_clean":"2024-09-12T02:18:06.455867+0000","last_became_active":"2024-09-12T02:16:29.124798+0000","last_became_peered":"2024-09-12T02:16:29.124798+0000","last_unstale":"2024-09-12T02:18:06.455867+0000","last_undegraded":"2024-09-12T02:18:06.455867+0000","last_fullsized":"2024-09-12T02:18:06.455867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:40:49.987718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.739010+0000","last_change":"2024-09-12T02:16:27.089668+0000","last_active":"2024-09-12T02:16:39.739010+0000","last_peered":"2024-09-12T02:16:39.739010+0000","last_clean":"2024-09-12T02:16:39.739010+0000","last_became_active":"2024-09-12T02:16:27.089350+0000","last_became_peered":"2024-09-12T02:16:27.089350+0000","last_unstale":"2024-09-12T02:16:39.739010+0000","last_undegraded":"2024-09-12T02:16:39.739010+0000","last_fullsized":"2024-09-12T02:16:39.739010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:35.528461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464409+0000","last_change":"2024-09-12T02:16:29.126544+0000","last_active":"2024-09-12T02:18:06.464409+0000","last_peered":"2024-09-12T02:18:06.464409+0000","last_clean":"2024-09-12T02:18:06.464409+0000","last_became_active":"2024-09-12T02:16:29.125669+0000","last_became_peered":"2024-09-12T02:16:29.125669+0000","last_unstale":"2024-09-12T02:18:06.464409+0000","last_undegraded":"2024-09-12T02:18:06.464409+0000","last_fullsized":"2024-09-12T02:18:06.464409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:20:19.415748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.893211+0000","last_change":"2024-09-12T02:16:27.549358+0000","last_active":"2024-09-12T02:18:29.893211+0000","last_peered":"2024-09-12T02:18:29.893211+0000","last_clean":"2024-09-12T02:18:29.893211+0000","last_became_active":"2024-09-12T02:16:27.548890+0000","last_became_peered":"2024-09-12T02:16:27.548890+0000","last_unstale":"2024-09-12T02:18:29.893211+0000","last_undegraded":"2024-09-12T02:18:29.893211+0000","last_fullsized":"2024-09-12T02:18:29.893211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:17.835178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459671+0000","last_change":"2024-09-12T02:16:29.126930+0000","last_active":"2024-09-12T02:18:06.459671+0000","last_peered":"2024-09-12T02:18:06.459671+0000","last_clean":"2024-09-12T02:18:06.459671+0000","last_became_active":"2024-09-12T02:16:29.125271+0000","last_became_peered":"2024-09-12T02:16:29.125271+0000","last_unstale":"2024-09-12T02:18:06.459671+0000","last_undegraded":"2024-09-12T02:18:06.459671+0000","last_fullsized":"2024-09-12T02:18:06.459671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:13:43.430581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.137892+0000","last_change":"2024-09-12T02:15:17.463704+0000","last_active":"2024-09-12T02:16:38.137892+0000","last_peered":"2024-09-12T02:16:38.137892+0000","last_clean":"2024-09-12T02:16:38.137892+0000","last_became_active":"2024-09-12T02:15:17.463226+0000","last_became_peered":"2024-09-12T02:15:17.463226+0000","last_unstale":"2024-09-12T02:16:38.137892+0000","last_undegraded":"2024-09-12T02:16:38.137892+0000","last_fullsized":"2024-09-12T02:16:38.137892+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:11:53.037848+0000","last_clean_scrub_stamp":"2024-09-12T02:11:53.037848+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:59.560882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:40.746915+0000","last_change":"2024-09-12T02:16:27.090365+0000","last_active":"2024-09-12T02:16:40.746915+0000","last_peered":"2024-09-12T02:16:40.746915+0000","last_clean":"2024-09-12T02:16:40.746915+0000","last_became_active":"2024-09-12T02:16:27.090008+0000","last_became_peered":"2024-09-12T02:16:27.090008+0000","last_unstale":"2024-09-12T02:16:40.746915+0000","last_undegraded":"2024-09-12T02:16:40.746915+0000","last_fullsized":"2024-09-12T02:16:40.746915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:33.687667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460904+0000","last_change":"2024-09-12T02:16:29.126598+0000","last_active":"2024-09-12T02:18:06.460904+0000","last_peered":"2024-09-12T02:18:06.460904+0000","last_clean":"2024-09-12T02:18:06.460904+0000","last_became_active":"2024-09-12T02:16:29.126141+0000","last_became_peered":"2024-09-12T02:16:29.126141+0000","last_unstale":"2024-09-12T02:18:06.460904+0000","last_undegraded":"2024-09-12T02:18:06.460904+0000","last_fullsized":"2024-09-12T02:18:06.460904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:10:06.050133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747039+0000","last_change":"2024-09-12T02:16:27.093273+0000","last_active":"2024-09-12T02:16:39.747039+0000","last_peered":"2024-09-12T02:16:39.747039+0000","last_clean":"2024-09-12T02:16:39.747039+0000","last_became_active":"2024-09-12T02:16:27.092853+0000","last_became_peered":"2024-09-12T02:16:27.092853+0000","last_unstale":"2024-09-12T02:16:39.747039+0000","last_undegraded":"2024-09-12T02:16:39.747039+0000","last_fullsized":"2024-09-12T02:16:39.747039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:13:42.173381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.460184+0000","last_change":"2024-09-12T02:16:29.129868+0000","last_active":"2024-09-12T02:18:06.460184+0000","last_peered":"2024-09-12T02:18:06.460184+0000","last_clean":"2024-09-12T02:18:06.460184+0000","last_became_active":"2024-09-12T02:16:29.129517+0000","last_became_peered":"2024-09-12T02:16:29.129517+0000","last_unstale":"2024-09-12T02:18:06.460184+0000","last_undegraded":"2024-09-12T02:18:06.460184+0000","last_fullsized":"2024-09-12T02:18:06.460184+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:37.366946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:31.894855+0000","last_change":"2024-09-12T02:16:27.116061+0000","last_active":"2024-09-12T02:18:31.894855+0000","last_peered":"2024-09-12T02:18:31.894855+0000","last_clean":"2024-09-12T02:18:31.894855+0000","last_became_active":"2024-09-12T02:16:27.115738+0000","last_became_peered":"2024-09-12T02:16:27.115738+0000","last_unstale":"2024-09-12T02:18:31.894855+0000","last_undegraded":"2024-09-12T02:18:31.894855+0000","last_fullsized":"2024-09-12T02:18:31.894855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:07:41.230143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447635+0000","last_change":"2024-09-12T02:16:29.129171+0000","last_active":"2024-09-12T02:18:06.447635+0000","last_peered":"2024-09-12T02:18:06.447635+0000","last_clean":"2024-09-12T02:18:06.447635+0000","last_became_active":"2024-09-12T02:16:29.128847+0000","last_became_peered":"2024-09-12T02:16:29.128847+0000","last_unstale":"2024-09-12T02:18:06.447635+0000","last_undegraded":"2024-09-12T02:18:06.447635+0000","last_fullsized":"2024-09-12T02:18:06.447635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:15.479474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:32.896333+0000","last_change":"2024-09-12T02:16:27.090267+0000","last_active":"2024-09-12T02:18:32.896333+0000","last_peered":"2024-09-12T02:18:32.896333+0000","last_clean":"2024-09-12T02:18:32.896333+0000","last_became_active":"2024-09-12T02:16:27.089864+0000","last_became_peered":"2024-09-12T02:16:27.089864+0000","last_unstale":"2024-09-12T02:18:32.896333+0000","last_undegraded":"2024-09-12T02:18:32.896333+0000","last_fullsized":"2024-09-12T02:18:32.896333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:04.489515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.438689+0000","last_change":"2024-09-12T02:16:29.133611+0000","last_active":"2024-09-12T02:18:06.438689+0000","last_peered":"2024-09-12T02:18:06.438689+0000","last_clean":"2024-09-12T02:18:06.438689+0000","last_became_active":"2024-09-12T02:16:29.132867+0000","last_became_peered":"2024-09-12T02:16:29.132867+0000","last_unstale":"2024-09-12T02:18:06.438689+0000","last_undegraded":"2024-09-12T02:18:06.438689+0000","last_fullsized":"2024-09-12T02:18:06.438689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:18:04.416961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.163130+0000","last_change":"2024-09-12T02:16:38.160740+0000","last_active":"2024-09-12T02:16:38.163130+0000","last_peered":"2024-09-12T02:16:38.163130+0000","last_clean":"2024-09-12T02:16:38.163130+0000","last_became_active":"2024-09-12T02:16:38.160372+0000","last_became_peered":"2024-09-12T02:16:38.160372+0000","last_unstale":"2024-09-12T02:16:38.163130+0000","last_undegraded":"2024-09-12T02:16:38.163130+0000","last_fullsized":"2024-09-12T02:16:38.163130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:12:47.412931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.447446+0000","last_change":"2024-09-12T02:16:29.131355+0000","last_active":"2024-09-12T02:18:06.447446+0000","last_peered":"2024-09-12T02:18:06.447446+0000","last_clean":"2024-09-12T02:18:06.447446+0000","last_became_active":"2024-09-12T02:16:29.131015+0000","last_became_peered":"2024-09-12T02:16:29.131015+0000","last_unstale":"2024-09-12T02:18:06.447446+0000","last_undegraded":"2024-09-12T02:18:06.447446+0000","last_fullsized":"2024-09-12T02:18:06.447446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:48:16.439741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.745709+0000","last_change":"2024-09-12T02:16:27.117261+0000","last_active":"2024-09-12T02:16:39.745709+0000","last_peered":"2024-09-12T02:16:39.745709+0000","last_clean":"2024-09-12T02:16:39.745709+0000","last_became_active":"2024-09-12T02:16:27.116357+0000","last_became_peered":"2024-09-12T02:16:27.116357+0000","last_unstale":"2024-09-12T02:16:39.745709+0000","last_undegraded":"2024-09-12T02:16:39.745709+0000","last_fullsized":"2024-09-12T02:16:39.745709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:51.379561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459968+0000","last_change":"2024-09-12T02:16:29.120854+0000","last_active":"2024-09-12T02:18:06.459968+0000","last_peered":"2024-09-12T02:18:06.459968+0000","last_clean":"2024-09-12T02:18:06.459968+0000","last_became_active":"2024-09-12T02:16:29.120447+0000","last_became_peered":"2024-09-12T02:16:29.120447+0000","last_unstale":"2024-09-12T02:18:06.459968+0000","last_undegraded":"2024-09-12T02:18:06.459968+0000","last_fullsized":"2024-09-12T02:18:06.459968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:13.966610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'17","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:21.137207+0000","last_change":"2024-09-12T02:16:38.165824+0000","last_active":"2024-09-12T02:18:21.137207+0000","last_peered":"2024-09-12T02:18:21.137207+0000","last_clean":"2024-09-12T02:18:21.137207+0000","last_became_active":"2024-09-12T02:16:38.160320+0000","last_became_peered":"2024-09-12T02:16:38.160320+0000","last_unstale":"2024-09-12T02:18:21.137207+0000","last_undegraded":"2024-09-12T02:18:21.137207+0000","last_fullsized":"2024-09-12T02:18:21.137207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:07:55.991191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464096+0000","last_change":"2024-09-12T02:16:29.124145+0000","last_active":"2024-09-12T02:18:06.464096+0000","last_peered":"2024-09-12T02:18:06.464096+0000","last_clean":"2024-09-12T02:18:06.464096+0000","last_became_active":"2024-09-12T02:16:29.123823+0000","last_became_peered":"2024-09-12T02:16:29.123823+0000","last_unstale":"2024-09-12T02:18:06.464096+0000","last_undegraded":"2024-09-12T02:18:06.464096+0000","last_fullsized":"2024-09-12T02:18:06.464096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:12.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747621+0000","last_change":"2024-09-12T02:16:27.549321+0000","last_active":"2024-09-12T02:16:39.747621+0000","last_peered":"2024-09-12T02:16:39.747621+0000","last_clean":"2024-09-12T02:16:39.747621+0000","last_became_active":"2024-09-12T02:16:27.549032+0000","last_became_peered":"2024-09-12T02:16:27.549032+0000","last_unstale":"2024-09-12T02:16:39.747621+0000","last_undegraded":"2024-09-12T02:16:39.747621+0000","last_fullsized":"2024-09-12T02:16:39.747621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:16:23.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.458006+0000","last_change":"2024-09-12T02:16:38.161245+0000","last_active":"2024-09-12T02:18:06.458006+0000","last_peered":"2024-09-12T02:18:06.458006+0000","last_clean":"2024-09-12T02:18:06.458006+0000","last_became_active":"2024-09-12T02:16:38.160895+0000","last_became_peered":"2024-09-12T02:16:38.160895+0000","last_unstale":"2024-09-12T02:18:06.458006+0000","last_undegraded":"2024-09-12T02:18:06.458006+0000","last_fullsized":"2024-09-12T02:18:06.458006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:47:04.795482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.135798+0000","last_change":"2024-09-12T02:16:27.116754+0000","last_active":"2024-09-12T02:16:38.135798+0000","last_peered":"2024-09-12T02:16:38.135798+0000","last_clean":"2024-09-12T02:16:38.135798+0000","last_became_active":"2024-09-12T02:16:27.116425+0000","last_became_peered":"2024-09-12T02:16:27.116425+0000","last_unstale":"2024-09-12T02:16:38.135798+0000","last_undegraded":"2024-09-12T02:16:38.135798+0000","last_fullsized":"2024-09-12T02:16:38.135798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:38:05.338258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.413775+0000","last_change":"2024-09-12T02:16:29.123810+0000","last_active":"2024-09-12T02:18:06.413775+0000","last_peered":"2024-09-12T02:18:06.413775+0000","last_clean":"2024-09-12T02:18:06.413775+0000","last_became_active":"2024-09-12T02:16:29.123499+0000","last_became_peered":"2024-09-12T02:16:29.123499+0000","last_unstale":"2024-09-12T02:18:06.413775+0000","last_undegraded":"2024-09-12T02:18:06.413775+0000","last_fullsized":"2024-09-12T02:18:06.413775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:26.544929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747539+0000","last_change":"2024-09-12T02:16:27.088915+0000","last_active":"2024-09-12T02:16:39.747539+0000","last_peered":"2024-09-12T02:16:39.747539+0000","last_clean":"2024-09-12T02:16:39.747539+0000","last_became_active":"2024-09-12T02:16:27.088588+0000","last_became_peered":"2024-09-12T02:16:27.088588+0000","last_unstale":"2024-09-12T02:16:39.747539+0000","last_undegraded":"2024-09-12T02:16:39.747539+0000","last_fullsized":"2024-09-12T02:16:39.747539+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:58:21.416990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451220+0000","last_change":"2024-09-12T02:16:29.122258+0000","last_active":"2024-09-12T02:18:06.451220+0000","last_peered":"2024-09-12T02:18:06.451220+0000","last_clean":"2024-09-12T02:18:06.451220+0000","last_became_active":"2024-09-12T02:16:29.121953+0000","last_became_peered":"2024-09-12T02:16:29.121953+0000","last_unstale":"2024-09-12T02:18:06.451220+0000","last_undegraded":"2024-09-12T02:18:06.451220+0000","last_fullsized":"2024-09-12T02:18:06.451220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:41:31.618063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.170628+0000","last_change":"2024-09-12T02:16:27.091555+0000","last_active":"2024-09-12T02:16:39.170628+0000","last_peered":"2024-09-12T02:16:39.170628+0000","last_clean":"2024-09-12T02:16:39.170628+0000","last_became_active":"2024-09-12T02:16:27.090811+0000","last_became_peered":"2024-09-12T02:16:27.090811+0000","last_unstale":"2024-09-12T02:16:39.170628+0000","last_undegraded":"2024-09-12T02:16:39.170628+0000","last_fullsized":"2024-09-12T02:16:39.170628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:13:19.118180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.440325+0000","last_change":"2024-09-12T02:16:38.157735+0000","last_active":"2024-09-12T02:18:06.440325+0000","last_peered":"2024-09-12T02:18:06.440325+0000","last_clean":"2024-09-12T02:18:06.440325+0000","last_became_active":"2024-09-12T02:16:38.157350+0000","last_became_peered":"2024-09-12T02:16:38.157350+0000","last_unstale":"2024-09-12T02:18:06.440325+0000","last_undegraded":"2024-09-12T02:18:06.440325+0000","last_fullsized":"2024-09-12T02:18:06.440325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:37:28.377716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.140039+0000","last_change":"2024-09-12T02:16:27.090397+0000","last_active":"2024-09-12T02:16:38.140039+0000","last_peered":"2024-09-12T02:16:38.140039+0000","last_clean":"2024-09-12T02:16:38.140039+0000","last_became_active":"2024-09-12T02:16:27.088069+0000","last_became_peered":"2024-09-12T02:16:27.088069+0000","last_unstale":"2024-09-12T02:16:38.140039+0000","last_undegraded":"2024-09-12T02:16:38.140039+0000","last_fullsized":"2024-09-12T02:16:38.140039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:25:35.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.459203+0000","last_change":"2024-09-12T02:16:29.130382+0000","last_active":"2024-09-12T02:18:06.459203+0000","last_peered":"2024-09-12T02:18:06.459203+0000","last_clean":"2024-09-12T02:18:06.459203+0000","last_became_active":"2024-09-12T02:16:29.129416+0000","last_became_peered":"2024-09-12T02:16:29.129416+0000","last_unstale":"2024-09-12T02:18:06.459203+0000","last_undegraded":"2024-09-12T02:18:06.459203+0000","last_fullsized":"2024-09-12T02:18:06.459203+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:07:44.265847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.451980+0000","last_change":"2024-09-12T02:16:29.133876+0000","last_active":"2024-09-12T02:18:06.451980+0000","last_peered":"2024-09-12T02:18:06.451980+0000","last_clean":"2024-09-12T02:18:06.451980+0000","last_became_active":"2024-09-12T02:16:29.133592+0000","last_became_peered":"2024-09-12T02:16:29.133592+0000","last_unstale":"2024-09-12T02:18:06.451980+0000","last_undegraded":"2024-09-12T02:18:06.451980+0000","last_fullsized":"2024-09-12T02:18:06.451980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:21.499170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.747189+0000","last_change":"2024-09-12T02:16:27.087048+0000","last_active":"2024-09-12T02:16:39.747189+0000","last_peered":"2024-09-12T02:16:39.747189+0000","last_clean":"2024-09-12T02:16:39.747189+0000","last_became_active":"2024-09-12T02:16:27.086684+0000","last_became_peered":"2024-09-12T02:16:27.086684+0000","last_unstale":"2024-09-12T02:16:39.747189+0000","last_undegraded":"2024-09-12T02:16:39.747189+0000","last_fullsized":"2024-09-12T02:16:39.747189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:28:13.272190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.461563+0000","last_change":"2024-09-12T02:16:29.130905+0000","last_active":"2024-09-12T02:18:06.461563+0000","last_peered":"2024-09-12T02:18:06.461563+0000","last_clean":"2024-09-12T02:18:06.461563+0000","last_became_active":"2024-09-12T02:16:29.130567+0000","last_became_peered":"2024-09-12T02:16:29.130567+0000","last_unstale":"2024-09-12T02:18:06.461563+0000","last_undegraded":"2024-09-12T02:18:06.461563+0000","last_fullsized":"2024-09-12T02:18:06.461563+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:31.889255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:39.748230+0000","last_change":"2024-09-12T02:16:27.088732+0000","last_active":"2024-09-12T02:16:39.748230+0000","last_peered":"2024-09-12T02:16:39.748230+0000","last_clean":"2024-09-12T02:16:39.748230+0000","last_became_active":"2024-09-12T02:16:27.088357+0000","last_became_peered":"2024-09-12T02:16:27.088357+0000","last_unstale":"2024-09-12T02:16:39.748230+0000","last_undegraded":"2024-09-12T02:16:39.748230+0000","last_fullsized":"2024-09-12T02:16:39.748230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:04.072678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.456396+0000","last_change":"2024-09-12T02:16:29.127431+0000","last_active":"2024-09-12T02:18:06.456396+0000","last_peered":"2024-09-12T02:18:06.456396+0000","last_clean":"2024-09-12T02:18:06.456396+0000","last_became_active":"2024-09-12T02:16:29.126923+0000","last_became_peered":"2024-09-12T02:16:29.126923+0000","last_unstale":"2024-09-12T02:18:06.456396+0000","last_undegraded":"2024-09-12T02:18:06.456396+0000","last_fullsized":"2024-09-12T02:18:06.456396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:17:49.405474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:16:38.157535+0000","last_change":"2024-09-12T02:16:27.115738+0000","last_active":"2024-09-12T02:16:38.157535+0000","last_peered":"2024-09-12T02:16:38.157535+0000","last_clean":"2024-09-12T02:16:38.157535+0000","last_became_active":"2024-09-12T02:16:27.115377+0000","last_became_peered":"2024-09-12T02:16:27.115377+0000","last_unstale":"2024-09-12T02:16:38.157535+0000","last_undegraded":"2024-09-12T02:16:38.157535+0000","last_fullsized":"2024-09-12T02:16:38.157535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:58:53.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:06.464122+0000","last_change":"2024-09-12T02:16:29.125712+0000","last_active":"2024-09-12T02:18:06.464122+0000","last_peered":"2024-09-12T02:18:06.464122+0000","last_clean":"2024-09-12T02:18:06.464122+0000","last_became_active":"2024-09-12T02:16:29.125373+0000","last_became_peered":"2024-09-12T02:16:29.125373+0000","last_unstale":"2024-09-12T02:18:06.464122+0000","last_undegraded":"2024-09-12T02:18:06.464122+0000","last_fullsized":"2024-09-12T02:18:06.464122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:28.094350+0000","last_clean_scrub_stamp":"2024-09-12T02:16:28.094350+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:31:32.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:18:29.891913+0000","last_change":"2024-09-12T02:16:27.091838+0000","last_active":"2024-09-12T02:18:29.891913+0000","last_peered":"2024-09-12T02:18:29.891913+0000","last_clean":"2024-09-12T02:18:29.891913+0000","last_became_active":"2024-09-12T02:16:27.091486+0000","last_became_peered":"2024-09-12T02:16:27.091486+0000","last_unstale":"2024-09-12T02:18:29.891913+0000","last_undegraded":"2024-09-12T02:18:29.891913+0000","last_fullsized":"2024-09-12T02:18:29.891913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:16:26.068968+0000","last_clean_scrub_stamp":"2024-09-12T02:16:26.068968+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:37.749913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1160,"num_read_kb":662,"num_write":266,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":231579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":251013,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":438915,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":1100181,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438845,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":439223,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":1032585,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439096,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1031028,"data_compressed":42120,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":439124,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438387,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444116,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":509231,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":509545,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439901,"data_compressed":5793,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34689,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36327,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:18:35.976 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:36.629 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:36 smithi092 ceph-mon[36089]: pgmap v342: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T02:18:36.629 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:36 smithi092 ceph-mon[36089]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:36.629 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:36 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:36.685 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:36.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:36 smithi026 ceph-mon[35496]: pgmap v342: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T02:18:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:36 smithi026 ceph-mon[35496]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:36 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:36.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:36 smithi003 ceph-mon[31476]: pgmap v342: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T02:18:36.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:36 smithi003 ceph-mon[31476]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:18:36.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:36 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.163+0000 7f7da7c15640 1 -- 172.21.15.3:0/3036202343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da01049a0 msgr2=0x7f7da01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.163+0000 7f7da7c15640 1 --2- 172.21.15.3:0/3036202343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da01049a0 0x7f7da01089f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009900 tx=0x7f7d9002a410 comp rx=0 tx=0).stop 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 -- 172.21.15.3:0/3036202343 shutdown_connections 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 --2- 172.21.15.3:0/3036202343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0109120 0x7f7da0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 --2- 172.21.15.3:0/3036202343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da01049a0 0x7f7da01089f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 --2- 172.21.15.3:0/3036202343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da0103ff0 0x7f7da01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 -- 172.21.15.3:0/3036202343 >> 172.21.15.3:0/3036202343 conn(0x7f7da00ffa90 msgr2=0x7f7da0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 -- 172.21.15.3:0/3036202343 shutdown_connections 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 -- 172.21.15.3:0/3036202343 wait complete. 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 Processor -- start 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.164+0000 7f7da7c15640 1 -- start start 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da01049a0 0x7f7da019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 0x7f7da019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0114120 con 0x7f7da0109120 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7da0113fa0 con 0x7f7da0103ff0 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01142a0 con 0x7f7da01049a0 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:37.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:53812/0 (socket says 172.21.15.3:53812) 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da61b2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 0x7f7da019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 -- 172.21.15.3:0/2808612308 learned_addr learned my addr 172.21.15.3:0/2808612308 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da51b0640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da01049a0 0x7f7da019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 -- 172.21.15.3:0/2808612308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da01049a0 msgr2=0x7f7da019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da01049a0 0x7f7da019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 -- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 msgr2=0x7f7da019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 0x7f7da019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 -- 172.21.15.3:0/2808612308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da019f650 con 0x7f7da0103ff0 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da61b2640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 0x7f7da019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da59b1640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d950 tx=0x7f7d9c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c017030 con 0x7f7da0103ff0 2024-09-12T02:18:37.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.165+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da019f940 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.166+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d9c004020 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.166+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c0047a0 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.166+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a7220 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.166+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da01043d0 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.168+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f7d9c0041d0 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.169+0000 7f7d96ffd640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 0x7f7d8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.169+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7d9c0cedd0 con 0x7f7da0103ff0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.169+0000 7f7da51b0640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 0x7f7d8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.169+0000 7f7da51b0640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 0x7f7d8c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009230 tx=0x7f7d900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:37.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.172+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d9c09bba0 con 0x7f7da0103ff0 2024-09-12T02:18:37.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.421+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da019ba00 con 0x7f7da0103ff0 2024-09-12T02:18:37.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.421+0000 7f7d96ffd640 1 -- 172.21.15.3:0/2808612308 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f7d9c09b4a0 con 0x7f7da0103ff0 2024-09-12T02:18:37.422 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:37.422 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:18:37.422 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:18:37.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.424+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 msgr2=0x7f7d8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.424+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 0x7f7d8c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009230 tx=0x7f7d900058d0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 msgr2=0x7f7da019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00d950 tx=0x7f7d9c00de10 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 shutdown_connections 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7d8c0a2c10 0x7f7d8c0a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7da0109120 0x7f7da019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7da01049a0 0x7f7da019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 --2- 172.21.15.3:0/2808612308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da0103ff0 0x7f7da019a6a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 >> 172.21.15.3:0/2808612308 conn(0x7f7da00ffa90 msgr2=0x7f7da0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 shutdown_connections 2024-09-12T02:18:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:37.425+0000 7f7da7c15640 1 -- 172.21.15.3:0/2808612308 wait complete. 2024-09-12T02:18:38.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:38.373 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:38.495 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: pgmap v343: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:18:38.495 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.495 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.495 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.496 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.496 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:38 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2808612308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: pgmap v343: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:38 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2808612308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: pgmap v343: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:38 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2808612308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 -- 172.21.15.3:0/968935535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 msgr2=0x7f610c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 --2- 172.21.15.3:0/968935535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c10ea90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6104009ca0 tx=0x7f610402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 -- 172.21.15.3:0/968935535 shutdown_connections 2024-09-12T02:18:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 --2- 172.21.15.3:0/968935535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c10ea90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 --2- 172.21.15.3:0/968935535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c102790 0x7f610c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 --2- 172.21.15.3:0/968935535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c101de0 0x7f610c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 -- 172.21.15.3:0/968935535 >> 172.21.15.3:0/968935535 conn(0x7f610c0fd8c0 msgr2=0x7f610c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 -- 172.21.15.3:0/968935535 shutdown_connections 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.874+0000 7f6114367640 1 -- 172.21.15.3:0/968935535 wait complete. 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 Processor -- start 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 -- start start 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c101de0 0x7f610c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c102790 0x7f610c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f610c111f70 con 0x7f610c106f10 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f610c111df0 con 0x7f610c102790 2024-09-12T02:18:38.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6114367640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f610c1120f0 con 0x7f610c101de0 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39998/0 (socket says 172.21.15.3:39998) 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 -- 172.21.15.3:0/1666043729 learned_addr learned my addr 172.21.15.3:0/1666043729 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112103640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c101de0 0x7f610c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6111902640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c102790 0x7f610c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 -- 172.21.15.3:0/1666043729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c101de0 msgr2=0x7f610c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c101de0 0x7f610c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 -- 172.21.15.3:0/1666043729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c102790 msgr2=0x7f610c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c102790 0x7f610c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 -- 172.21.15.3:0/1666043729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c19d3a0 con 0x7f610c106f10 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.875+0000 7f6112904640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6104031a20 tx=0x7f6104031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.876+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104003c30 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.876+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c19d630 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.876+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6104003dd0 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.876+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610402fcf0 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.876+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1a4f70 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.877+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c1021c0 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.878+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6104004050 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.879+0000 7f61037fe640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 0x7f60f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.879+0000 7f6112103640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 0x7f60f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.879+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f61040f0550 con 0x7f610c106f10 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.880+0000 7f6112103640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 0x7f60f80a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f610800bbe0 tx=0x7f610800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:38.882+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61040bd4e0 con 0x7f610c106f10 2024-09-12T02:18:39.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.094+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f610c10a0f0 con 0x7f610c106f10 2024-09-12T02:18:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.095+0000 7f61037fe640 1 -- 172.21.15.3:0/1666043729 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f610402fe90 con 0x7f610c106f10 2024-09-12T02:18:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:18:39.095 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:39.096 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:18:39.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 msgr2=0x7f60f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 0x7f60f80a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f610800bbe0 tx=0x7f610800b390 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 msgr2=0x7f610c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6104031a20 tx=0x7f6104031a50 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 shutdown_connections 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60f80a2c10 0x7f60f80a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f610c106f10 0x7f610c19ccc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f610c102790 0x7f610c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 --2- 172.21.15.3:0/1666043729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f610c101de0 0x7f610c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.097+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 >> 172.21.15.3:0/1666043729 conn(0x7f610c0fd8c0 msgr2=0x7f610c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.098+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 shutdown_connections 2024-09-12T02:18:39.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:39.098+0000 7f6114367640 1 -- 172.21.15.3:0/1666043729 wait complete. 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:39 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1666043729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:39 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1666043729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:39.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:18:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:18:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:18:39.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:39 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1666043729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:39.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:18:40.056 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:40.473 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:40 smithi003 ceph-mon[31476]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:18:40.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:40 smithi026 ceph-mon[35496]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:18:40.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:40 smithi092 ceph-mon[36089]: pgmap v344: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:18:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 -- 172.21.15.3:0/1019615525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534103640 msgr2=0x7f753410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1019615525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534103640 0x7f753410bb30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7528009900 tx=0x7f752802a480 comp rx=0 tx=0).stop 2024-09-12T02:18:40.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 -- 172.21.15.3:0/1019615525 shutdown_connections 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1019615525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f753410c150 0x7f753410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1019615525 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7534103640 0x7f753410bb30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1019615525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 0x7f7534103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.556+0000 7f753a6cd640 1 -- 172.21.15.3:0/1019615525 >> 172.21.15.3:0/1019615525 conn(0x7f75340fc780 msgr2=0x7f75340feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- 172.21.15.3:0/1019615525 shutdown_connections 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- 172.21.15.3:0/1019615525 wait complete. 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 Processor -- start 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- start start 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 0x7f7534198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7534103640 0x7f7534198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:40.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7534111fe0 con 0x7f753410c150 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7534111e60 con 0x7f7534102d20 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.557+0000 7f753a6cd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7534112160 con 0x7f7534103640 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40004/0 (socket says 172.21.15.3:40004) 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 -- 172.21.15.3:0/1560425474 learned_addr learned my addr 172.21.15.3:0/1560425474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7533fff640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 0x7f7534198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f75337fe640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7534103640 0x7f7534198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 -- 172.21.15.3:0/1560425474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7534103640 msgr2=0x7f7534198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7534103640 0x7f7534198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 -- 172.21.15.3:0/1560425474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 msgr2=0x7f7534198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:40.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 0x7f7534198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 -- 172.21.15.3:0/1560425474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753419d470 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f7538c6a640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b970 tx=0x7f752c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c019070 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c00cb40 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753419d760 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c007650 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.558+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a5040 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.559+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7534103100 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.561+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f752c01c070 con 0x7f753410c150 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.561+0000 7f75317fa640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.561+0000 7f7533fff640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:40.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.561+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f752c0ce770 con 0x7f753410c150 2024-09-12T02:18:40.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.562+0000 7f7533fff640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 0x7f751c0a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7524005fd0 tx=0x7f752400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:40.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.566+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f752c09b670 con 0x7f753410c150 2024-09-12T02:18:40.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.742+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7534106c30 con 0x7f753410c150 2024-09-12T02:18:40.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.744+0000 7f75317fa640 1 -- 172.21.15.3:0/1560425474 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f752c09af70 con 0x7f753410c150 2024-09-12T02:18:40.744 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:40.744 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 msgr2=0x7f751c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 0x7f751c0a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7524005fd0 tx=0x7f752400b590 comp rx=0 tx=0).stop 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 msgr2=0x7f753419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f752c00b970 tx=0x7f752c00be30 comp rx=0 tx=0).stop 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 shutdown_connections 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f751c0a2c10 0x7f751c0a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f753410c150 0x7f753419cd90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7534103640 0x7f7534198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 --2- 172.21.15.3:0/1560425474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7534102d20 0x7f7534198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:40.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 >> 172.21.15.3:0/1560425474 conn(0x7f75340fc780 msgr2=0x7f75340fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:40.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 shutdown_connections 2024-09-12T02:18:40.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:40.746+0000 7f753a6cd640 1 -- 172.21.15.3:0/1560425474 wait complete. 2024-09-12T02:18:41.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:41 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:18:41.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:41 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1560425474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:41.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T02:18:41.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:41 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:18:41.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:41 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1560425474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:41.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:41 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:18:41.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:41 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1560425474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:18:41.708 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 -- 172.21.15.3:0/365776754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8104f20 msgr2=0x7f7de8107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 --2- 172.21.15.3:0/365776754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8104f20 0x7f7de8107310 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8009900 tx=0x7f7dd802a410 comp rx=0 tx=0).stop 2024-09-12T02:18:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 -- 172.21.15.3:0/365776754 shutdown_connections 2024-09-12T02:18:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 --2- 172.21.15.3:0/365776754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de8107850 0x7f7de8109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 --2- 172.21.15.3:0/365776754 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8104f20 0x7f7de8107310 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 --2- 172.21.15.3:0/365776754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de81025f0 0x7f7de81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 -- 172.21.15.3:0/365776754 >> 172.21.15.3:0/365776754 conn(0x7f7de80fc1f0 msgr2=0x7f7de80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 -- 172.21.15.3:0/365776754 shutdown_connections 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.281+0000 7f7df072c640 1 -- 172.21.15.3:0/365776754 wait complete. 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 Processor -- start 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 -- start start 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de81025f0 0x7f7de8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 0x7f7de8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d820 con 0x7f7de8107850 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d6a0 con 0x7f7de81025f0 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7df072c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7de810d9a0 con 0x7f7de8104f20 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40022/0 (socket says 172.21.15.3:40022) 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7deecc9640 1 -- 172.21.15.3:0/3176448073 learned_addr learned my addr 172.21.15.3:0/3176448073 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de81025f0 0x7f7de8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.282+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 0x7f7de8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 -- 172.21.15.3:0/3176448073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 msgr2=0x7f7de8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 0x7f7de8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 -- 172.21.15.3:0/3176448073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de81025f0 msgr2=0x7f7de8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de81025f0 0x7f7de8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 -- 172.21.15.3:0/3176448073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de819fc10 con 0x7f7de8107850 2024-09-12T02:18:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 0x7f7de8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e9d0 tx=0x7f7de000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7ddf7fe640 1 -- 172.21.15.3:0/3176448073 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de000cd80 con 0x7f7de8107850 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7ddf7fe640 1 -- 172.21.15.3:0/3176448073 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de0005020 con 0x7f7de8107850 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de819ff00 con 0x7f7de8107850 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7ddf7fe640 1 -- 172.21.15.3:0/3176448073 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00116d0 con 0x7f7de8107850 2024-09-12T02:18:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.283+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81a03c0 con 0x7f7de8107850 2024-09-12T02:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.285+0000 7f7ddf7fe640 1 -- 172.21.15.3:0/3176448073 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f7de00051d0 con 0x7f7de8107850 2024-09-12T02:18:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.285+0000 7f7df34c2740 1 Processor -- start 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 -- start start 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fc9a0 0x55909b7fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 0x55909b801da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b804830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55909b7602f0 con 0x55909b802370 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55909b708270 con 0x55909b7fc9a0 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7df34c2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55909b75c2b0 con 0x55909b7fd2e0 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dedcc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b804830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dedcc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b804830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40034/0 (socket says 172.21.15.3:40034) 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dedcc7640 1 -- 172.21.15.3:0/2858981030 learned_addr learned my addr 172.21.15.3:0/2858981030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 0x55909b801da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fc9a0 0x55909b7fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 -- 172.21.15.3:0/2858981030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fc9a0 msgr2=0x55909b7fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fc9a0 0x55909b7fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 -- 172.21.15.3:0/2858981030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 msgr2=0x55909b804830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b804830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7dee4c8640 1 -- 172.21.15.3:0/2858981030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909b707340 con 0x55909b7fd2e0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.286+0000 7f7ddf7fe640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 0x7f7dd40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 0x7f7dd40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 0x55909b801da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7de400ab40 tx=0x7f7de400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a8fbc689b116af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7ddf7fe640 1 -- 172.21.15.3:0/3176448073 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7de00cf6b0 con 0x7f7de8107850 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7ddcff9640 1 -- 172.21.15.3:0/2858981030 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4004020 con 0x55909b7fd2e0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7ddcff9640 1 -- 172.21.15.3:0/2858981030 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de40041c0 con 0x55909b7fd2e0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.287+0000 7f7ddcff9640 1 -- 172.21.15.3:0/2858981030 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4018750 con 0x55909b7fd2e0 2024-09-12T02:18:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- 172.21.15.3:0/2858981030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 msgr2=0x55909b801da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 0x7f7dd40a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005ec0 tx=0x7f7dd8040040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 0x55909b801da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7de400ab40 tx=0x7f7de400f3c0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- 172.21.15.3:0/2858981030 shutdown_connections 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b804830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fd2e0 0x55909b801da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2858981030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fc9a0 0x55909b7fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- 172.21.15.3:0/2858981030 >> 172.21.15.3:0/2858981030 conn(0x55909b7f8010 msgr2=0x55909b7f8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- 172.21.15.3:0/2858981030 shutdown_connections 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- 172.21.15.3:0/2858981030 wait complete. 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- start start 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 Processor -- start 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b806b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b807070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55909b805340 con 0x55909b802370 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55909b6e9290 con 0x55909b7fd2e0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55909b8054c0 con 0x55909b7fc9a0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.288+0000 7f7df34c2740 10 client.0 _ll_register_callbacks cb 0x55909b71a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40038/0 (socket says 172.21.15.3:40038) 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 -- 172.21.15.3:0/2792194043 learned_addr learned my addr 172.21.15.3:0/2792194043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b806b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b807070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 -- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 msgr2=0x55909b806b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b806b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 -- 172.21.15.3:0/2792194043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 msgr2=0x55909b807070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b807070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 -- 172.21.15.3:0/2792194043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909b473210 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b807070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7deecc9640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005ba0 tx=0x7f7de009c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6d6150266d2037 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de009f070 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7de00a4580 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de40041c0 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7de001f070 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7df34c2740 10 client.15375 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55909b53f620 con 0x55909b802370 2024-09-12T02:18:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.289+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de40041c0 con 0x55909b802370 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7de000f040 con 0x55909b802370 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 10 client.15375 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 20 client.15375 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b812800 0x55909b814c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 4 client.15375 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909b76da20 con 0x55909b812800 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b812800 0x55909b814c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7dd1ffb640 10 client.15375 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.290+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b812800 0x55909b814c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cede94c9b40519 server_cookie=6fab052ecb184760 in_seq=0 out_seq=0 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/2792194043 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55909b76da20 con 0x55909b812800 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7dd1ffb640 10 client.15375 handle_command_reply: tid=0 2024-09-12T02:18:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 client.15375 shutdown 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 msgr2=0x55909b80e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7de0005ba0 tx=0x7f7de009c5d0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 shutdown_connections 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b812800 0x55909b814c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b80e0b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b807070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7dd1ffb640 0 client.15375 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 --2- 172.21.15.3:0/2792194043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b806b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 >> 172.21.15.3:0/2792194043 conn(0x55909b7f8010 msgr2=0x55909b7f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 shutdown_connections 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.291+0000 7f7df34c2740 1 -- 172.21.15.3:0/2792194043 wait complete. 2024-09-12T02:18:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.292+0000 7f7df34c2740 20 client.15375 trim_cache size 0 max 16384 2024-09-12T02:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 Processor -- start 2024-09-12T02:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 -- start start 2024-09-12T02:18:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b7f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81a90e0 con 0x55909b802370 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81aac20 con 0x55909b7fd2e0 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7df34c2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81a97b0 con 0x55909b7fc9a0 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7deecc9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40040/0 (socket says 172.21.15.3:40040) 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.306+0000 7f7deecc9640 1 -- 172.21.15.3:0/3504303759 learned_addr learned my addr 172.21.15.3:0/3504303759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b7f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 -- 172.21.15.3:0/3504303759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 msgr2=0x55909b7f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b7f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 -- 172.21.15.3:0/3504303759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 msgr2=0x55909b80e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 -- 172.21.15.3:0/3504303759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909b7f7e30 con 0x55909b802370 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7deecc9640 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7de00a44e0 tx=0x7f7de00056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63d7351ffca0a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7dd3fff640 1 -- 172.21.15.3:0/3504303759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de009f040 con 0x55909b802370 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7dd3fff640 1 -- 172.21.15.3:0/3504303759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de009adb0 con 0x55909b802370 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.307+0000 7f7dd3fff640 1 -- 172.21.15.3:0/3504303759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00b9080 con 0x55909b802370 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- 172.21.15.3:0/3504303759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 msgr2=0x55909b7f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7de00a44e0 tx=0x7f7de00056f0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- 172.21.15.3:0/3504303759 shutdown_connections 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b802370 0x55909b7f9400 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- 172.21.15.3:0/3504303759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b7fc9a0 0x55909b7f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- 172.21.15.3:0/3504303759 >> 172.21.15.3:0/3504303759 conn(0x55909b7f8010 msgr2=0x55909b806b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- 172.21.15.3:0/3504303759 shutdown_connections 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- 172.21.15.3:0/3504303759 wait complete. 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- start start 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 Processor -- start 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 0x55909b80cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81aa550 con 0x55909b7fc9a0 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81a9e80 con 0x55909b7fd2e0 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81a70c0 con 0x55909b802370 2024-09-12T02:18:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7df34c2740 10 client.0 _ll_register_callbacks cb 0x7f7de0098040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7dee4c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7dee4c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40042/0 (socket says 172.21.15.3:40042) 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.308+0000 7f7dee4c8640 1 -- 172.21.15.3:0/1715358215 learned_addr learned my addr 172.21.15.3:0/1715358215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7deecc9640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 0x55909b80cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 -- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 msgr2=0x55909b80cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 0x55909b80cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 -- 172.21.15.3:0/1715358215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 msgr2=0x55909b80bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 -- 172.21.15.3:0/1715358215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909b80d500 con 0x55909b7fc9a0 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7deecc9640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 0x55909b80cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dee4c8640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7de40058c0 tx=0x7f7de4005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff305e87849d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de402f070 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7de40035f0 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de009adb0 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7de403b050 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7df34c2740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55909b87d750 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.309+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de009adb0 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7de4005400 con 0x55909b7fc9a0 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 20 client.15381 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b87de50 0x55909b880270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 4 client.15381 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:18:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55909b767d70 con 0x55909b87de50 2024-09-12T02:18:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b87de50 0x55909b880270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7dd1ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.310+0000 7f7dedcc7640 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b87de50 0x55909b880270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb096c8883791e8 server_cookie=aa37aa22b0299143 in_seq=0 out_seq=0 2024-09-12T02:18:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7dd1ffb640 1 -- 172.21.15.3:0/1715358215 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55909b767d70 con 0x55909b87de50 2024-09-12T02:18:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7dd1ffb640 10 client.15381 handle_command_reply: tid=0 2024-09-12T02:18:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 client.15381 shutdown 2024-09-12T02:18:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 msgr2=0x55909b80b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7de40058c0 tx=0x7f7de4005bf0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 shutdown_connections 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55909b87de50 0x55909b880270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55909b802370 0x55909b80cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55909b7fd2e0 0x55909b80bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 --2- 172.21.15.3:0/1715358215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55909b7fc9a0 0x55909b80b9f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7dd1ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 >> 172.21.15.3:0/1715358215 conn(0x55909b7f8010 msgr2=0x55909b7f69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 shutdown_connections 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 1 -- 172.21.15.3:0/1715358215 wait complete. 2024-09-12T02:18:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.321+0000 7f7df34c2740 20 client.15381 trim_cache size 0 max 16384 2024-09-12T02:18:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:18:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:18:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:18:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 msgr2=0x7f7dd40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 0x7f7dd40a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005ec0 tx=0x7f7dd8040040 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 msgr2=0x7f7de819f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7de000e9d0 tx=0x7f7de000ee90 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 shutdown_connections 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7dd40a2c90 0x7f7dd40a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7de8107850 0x7f7de819f5a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8104f20 0x7f7de8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 --2- 172.21.15.3:0/3176448073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7de81025f0 0x7f7de8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 >> 172.21.15.3:0/3176448073 conn(0x7f7de80fc1f0 msgr2=0x7f7de81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 shutdown_connections 2024-09-12T02:18:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:42.322+0000 7f7df072c640 1 -- 172.21.15.3:0/3176448073 wait complete. 2024-09-12T02:18:42.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:42 smithi026 ceph-mon[35496]: pgmap v345: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:18:42.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:42 smithi003 ceph-mon[31476]: pgmap v345: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:18:42.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:42 smithi092 ceph-mon[36089]: pgmap v345: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:18:42.868 INFO:teuthology.orchestra.run.smithi003.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T02:18:42.868 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T02:18:42.869 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T02:18:42.869 DEBUG:tasks.quiescer:Yielding 2024-09-12T02:18:42.869 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T02:18:42.884 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:18:42.885 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T02:18:42.885 INFO:tasks.workunit:timeout=3h 2024-09-12T02:18:42.885 INFO:tasks.workunit:cleanup=True 2024-09-12T02:18:42.885 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:42.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:18:42.903 INFO:teuthology.orchestra.run.smithi003.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:42.903 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-09-12 02:16:48.762771386 +0000 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-09-12 02:16:48.762771386 +0000 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-09-12 02:18:20.479611703 +0000 2024-09-12T02:18:42.904 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-09-12 02:16:48.762771386 +0000 2024-09-12T02:18:42.904 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T02:18:42.904 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T02:18:42.982 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:18:43.039 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T02:18:43.151 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.643+0000 7fb0dcd96640 1 -- 172.21.15.3:0/3451296190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81082a0 msgr2=0x7fb0d810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.643+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/3451296190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81082a0 0x7fb0d810ec40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009cd0 tx=0x7fb0d002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 -- 172.21.15.3:0/3451296190 shutdown_connections 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/3451296190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81082a0 0x7fb0d810ec40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/3451296190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81036e0 0x7fb0d8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/3451296190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 0x7fb0d8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 -- 172.21.15.3:0/3451296190 >> 172.21.15.3:0/3451296190 conn(0x7fb0d80fc7d0 msgr2=0x7fb0d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 -- 172.21.15.3:0/3451296190 shutdown_connections 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 -- 172.21.15.3:0/3451296190 wait complete. 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.644+0000 7fb0dcd96640 1 Processor -- start 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 -- start start 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 0x7fb0d8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81082a0 0x7fb0d8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d81120a0 con 0x7fb0d81036e0 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8111f20 con 0x7fb0d81082a0 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0dcd96640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d8112220 con 0x7fb0d8102d30 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40056/0 (socket says 172.21.15.3:40056) 2024-09-12T02:18:43.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 -- 172.21.15.3:0/1172000937 learned_addr learned my addr 172.21.15.3:0/1172000937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0d6d9d640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81082a0 0x7fb0d8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0d659c640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 0x7fb0d8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 -- 172.21.15.3:0/1172000937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 msgr2=0x7fb0d8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 0x7fb0d8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 -- 172.21.15.3:0/1172000937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81082a0 msgr2=0x7fb0d8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81082a0 0x7fb0d8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.645+0000 7fb0cdd9b640 1 -- 172.21.15.3:0/1172000937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d81a69d0 con 0x7fb0d81036e0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0cdd9b640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c970 tx=0x7fb0c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0007bb0 con 0x7fb0d81036e0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d81a6cc0 con 0x7fb0d81036e0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c0007d50 con 0x7fb0d81036e0 2024-09-12T02:18:43.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000f450 con 0x7fb0d81036e0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.646+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d81a71f0 con 0x7fb0d81036e0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.647+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d8103110 con 0x7fb0d81036e0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.648+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb0c0004f60 con 0x7fb0d81036e0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.649+0000 7fb0cffff640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 0x7fb0c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.649+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb0c00cea00 con 0x7fb0d81036e0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.649+0000 7fb0d659c640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 0x7fb0c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.650+0000 7fb0d659c640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 0x7fb0c40a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005d00 tx=0x7fb0c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:43.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.653+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0c00d0e20 con 0x7fb0d81036e0 2024-09-12T02:18:43.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.865+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0d810a2b0 con 0x7fb0d81036e0 2024-09-12T02:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.866+0000 7fb0cffff640 1 -- 172.21.15.3:0/1172000937 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7fb0c0014030 con 0x7fb0d81036e0 2024-09-12T02:18:43.866 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:18:43.866 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:18:43.866 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 msgr2=0x7fb0c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 0x7fb0c40a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8005d00 tx=0x7fb0c800b410 comp rx=0 tx=0).stop 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 msgr2=0x7fb0d8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c970 tx=0x7fb0c000ce30 comp rx=0 tx=0).stop 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 shutdown_connections 2024-09-12T02:18:43.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb0c40a2c10 0x7fb0c40a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0d81082a0 0x7fb0d8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0d81036e0 0x7fb0d8100ed0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 --2- 172.21.15.3:0/1172000937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d8102d30 0x7fb0d8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 >> 172.21.15.3:0/1172000937 conn(0x7fb0d80fc7d0 msgr2=0x7fb0d8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 shutdown_connections 2024-09-12T02:18:43.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:43.868+0000 7fb0dcd96640 1 -- 172.21.15.3:0/1172000937 wait complete. 2024-09-12T02:18:44.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:44 smithi026 ceph-mon[35496]: pgmap v346: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:44.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:44 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1172000937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:44.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:44 smithi003 ceph-mon[31476]: pgmap v346: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:44.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:44 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1172000937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:44.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:44 smithi092 ceph-mon[36089]: pgmap v346: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:18:44.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:44 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1172000937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:18:44.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:16:29.718750+0000', 'modified': '2024-09-12T02:18:21.173770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24508, 'mds_1': 24493, 'mds_2': 24487, 'mds_3': 24482, 'mds_4': 14619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14613': {'gid': 14613, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.3:6839/775355044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6838', 'nonce': 775355044}, {'type': 'v1', 'addr': '172.21.15.3:6839', 'nonce': 775355044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14619': {'gid': 14619, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.3:6841/3157578506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6840', 'nonce': 3157578506}, {'type': 'v1', 'addr': '172.21.15.3:6841', 'nonce': 3157578506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6833/1804609961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 1804609961}, {'type': 'v1', 'addr': '172.21.15.26:6833', 'nonce': 1804609961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1866420590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1866420590}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1866420590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/3877701831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 3877701831}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 3877701831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6837/2891911183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 2891911183}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 2891911183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6839/1997151172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 1997151172}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 1997151172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6835/1180218932', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1180218932}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 1180218932}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.92:6837/487477954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 487477954}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 487477954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.92:6839/1571545450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 1571545450}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 1571545450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24508, 'qdb_cluster': [24493, 24487, 24508, 24482, 14619]} 2024-09-12T02:18:44.565 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:18:44.565 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-12T02:18:44.866 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 -- 172.21.15.3:0/3064965212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e410c170 msgr2=0x7fc9e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/3064965212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e410c170 0x7fc9e410e610 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009ca0 tx=0x7fc9dc02a950 comp rx=0 tx=0).stop 2024-09-12T02:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 -- 172.21.15.3:0/3064965212 shutdown_connections 2024-09-12T02:18:45.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/3064965212 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e410c170 0x7fc9e410e610 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/3064965212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e4103660 0x7fc9e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/3064965212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4102d40 0x7fc9e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 -- 172.21.15.3:0/3064965212 >> 172.21.15.3:0/3064965212 conn(0x7fc9e40fc7a0 msgr2=0x7fc9e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 -- 172.21.15.3:0/3064965212 shutdown_connections 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.344+0000 7fc9ebc48640 1 -- 172.21.15.3:0/3064965212 wait complete. 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 Processor -- start 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 -- start start 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4103660 0x7fc9e4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e410c170 0x7fc9e4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4111f30 con 0x7fc9e4102d40 2024-09-12T02:18:45.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e4111db0 con 0x7fc9e410c170 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ebc48640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e41120b0 con 0x7fc9e4103660 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40082/0 (socket says 172.21.15.3:40082) 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 -- 172.21.15.3:0/1935320762 learned_addr learned my addr 172.21.15.3:0/1935320762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e410c170 0x7fc9e4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4103660 0x7fc9e4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 -- 172.21.15.3:0/1935320762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4103660 msgr2=0x7fc9e4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4103660 0x7fc9e4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 -- 172.21.15.3:0/1935320762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e410c170 msgr2=0x7fc9e4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e410c170 0x7fc9e4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.345+0000 7fc9e99e4640 1 -- 172.21.15.3:0/1935320762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e41a68d0 con 0x7fc9e4102d40 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d950 tx=0x7fc9e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9daffd640 1 -- 172.21.15.3:0/1935320762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0017070 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e41a6bc0 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9daffd640 1 -- 172.21.15.3:0/1935320762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0009c30 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9daffd640 1 -- 172.21.15.3:0/1935320762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000f6b0 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.346+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e41a7100 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.348+0000 7fc9daffd640 1 -- 172.21.15.3:0/1935320762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc9e00109d0 con 0x7fc9e4102d40 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.348+0000 7fc9ee021740 1 Processor -- start 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 -- start start 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a64670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5626559c0090 con 0x562655a599d0 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562655967a70 con 0x562655a64bb0 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ee021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626559bc050 con 0x562655a5a2f0 2024-09-12T02:18:45.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40088/0 (socket says 172.21.15.3:40088) 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/2550989566 learned_addr learned my addr 172.21.15.3:0/2550989566 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a64670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9daffd640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/2550989566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 msgr2=0x562655a64670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a64670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/2550989566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 msgr2=0x562655a66fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a66fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/2550989566 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562655974130 con 0x562655a599d0 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9daffd640 1 -- 172.21.15.3:0/1935320762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc9e00cee30 con 0x7fc9e4102d40 2024-09-12T02:18:45.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.349+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.350+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009ca0 tx=0x7fc9dc0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f9488b082ed6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.350+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc04d000 tx=0x7fc9dc04a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.350+0000 7fc9cffff640 1 -- 172.21.15.3:0/2550989566 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc034040 con 0x562655a599d0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.350+0000 7fc9cffff640 1 -- 172.21.15.3:0/2550989566 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc003c30 con 0x562655a599d0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.350+0000 7fc9cffff640 1 -- 172.21.15.3:0/2550989566 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0467d0 con 0x562655a599d0 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- 172.21.15.3:0/2550989566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 msgr2=0x562655a59db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009ca0 tx=0x7fc9dc0098f0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- 172.21.15.3:0/2550989566 shutdown_connections 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a66fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a64670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2550989566 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a59db0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- 172.21.15.3:0/2550989566 >> 172.21.15.3:0/2550989566 conn(0x562655a55f60 msgr2=0x562655a56340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- 172.21.15.3:0/2550989566 shutdown_connections 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- 172.21.15.3:0/2550989566 wait complete. 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 -- start start 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.351+0000 7fc9ee021740 1 Processor -- start 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 0x562655a644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a5a2f0 0x562655a617f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562655a67780 con 0x562655a64bb0 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56265597af20 con 0x562655a5a2f0 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562655a67900 con 0x562655a599d0 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 10 client.0 _ll_register_callbacks cb 0x562655944030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40090/0 (socket says 172.21.15.3:40090) 2024-09-12T02:18:45.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/3483088418 learned_addr learned my addr 172.21.15.3:0/3483088418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 0x562655a644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a5a2f0 0x562655a617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/3483088418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 msgr2=0x562655a644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 0x562655a644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/3483088418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a5a2f0 msgr2=0x562655a617f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a5a2f0 0x562655a617f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 -- 172.21.15.3:0/3483088418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626557312b0 con 0x562655a64bb0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 0x562655a644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0045c0 tx=0x7fc9dc0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a497a4cb2d42478 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc02fe10 con 0x562655a64bb0 2024-09-12T02:18:45.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc9dc046550 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.352+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9dc003c30 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc046790 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562655a67780 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc003c30 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9dc030c20 con 0x562655a64bb0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 10 client.15399 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 20 client.15399 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655a752c0 0x562655a76730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 4 client.15399 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626559cd600 con 0x562655a752c0 2024-09-12T02:18:45.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.353+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655a752c0 0x562655a76730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655a752c0 0x562655a76730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3d12c63a431189c server_cookie=a4993b11fd87ff3b in_seq=0 out_seq=0 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9cd7fa640 10 client.15399 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/3483088418 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626559cd600 con 0x562655a752c0 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9cd7fa640 10 client.15399 handle_command_reply: tid=0 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 client.15399 shutdown 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 msgr2=0x562655a61d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0045c0 tx=0x7fc9dc0045f0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 shutdown_connections 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a64bb0 0x562655a61d30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a5a2f0 0x562655a617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655a752c0 0x562655a76730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 --2- 172.21.15.3:0/3483088418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a599d0 0x562655a644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9cd7fa640 0 client.15399 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.354+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 >> 172.21.15.3:0/3483088418 conn(0x562655a55f60 msgr2=0x562655a5bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.355+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 shutdown_connections 2024-09-12T02:18:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.355+0000 7fc9ee021740 1 -- 172.21.15.3:0/3483088418 wait complete. 2024-09-12T02:18:45.370 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.355+0000 7fc9ee021740 20 client.15399 trim_cache size 0 max 16384 2024-09-12T02:18:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 Processor -- start 2024-09-12T02:18:45.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 -- start start 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a599d0 0x562655a74270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a64bb0 0x562655a5ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562655a74a70 con 0x562655a5a2f0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562655a75140 con 0x562655a599d0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ee021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562655a75810 con 0x562655a64bb0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40102/0 (socket says 172.21.15.3:40102) 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 -- 172.21.15.3:0/2124106054 learned_addr learned my addr 172.21.15.3:0/2124106054 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a599d0 0x562655a74270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a64bb0 0x562655a5ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.374 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 -- 172.21.15.3:0/2124106054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a64bb0 msgr2=0x562655a5ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a64bb0 0x562655a5ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 -- 172.21.15.3:0/2124106054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a599d0 msgr2=0x562655a74270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a599d0 0x562655a74270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.373+0000 7fc9e91e3640 1 -- 172.21.15.3:0/2124106054 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562655a64810 con 0x562655a5a2f0 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4102430 tx=0x7fc9d4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1654802eb2057325 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9cf7fe640 1 -- 172.21.15.3:0/2124106054 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400bca0 con 0x562655a5a2f0 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9cf7fe640 1 -- 172.21.15.3:0/2124106054 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d400be40 con 0x562655a5a2f0 2024-09-12T02:18:45.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9cf7fe640 1 -- 172.21.15.3:0/2124106054 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d401b450 con 0x562655a5a2f0 2024-09-12T02:18:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 -- 172.21.15.3:0/2124106054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 msgr2=0x562655a54ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4102430 tx=0x7fc9d4012820 comp rx=0 tx=0).stop 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 -- 172.21.15.3:0/2124106054 shutdown_connections 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a64bb0 0x562655a5ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a5a2f0 0x562655a54ef0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2124106054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a599d0 0x562655a74270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.374+0000 7fc9ee021740 1 -- 172.21.15.3:0/2124106054 >> 172.21.15.3:0/2124106054 conn(0x562655a55f60 msgr2=0x562655a547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- 172.21.15.3:0/2124106054 shutdown_connections 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- 172.21.15.3:0/2124106054 wait complete. 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- start start 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 Processor -- start 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a5ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562655a765b0 con 0x562655a599d0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562655a75ee0 con 0x562655a64bb0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562655a76c80 con 0x562655a5a2f0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9ee021740 10 client.0 _ll_register_callbacks cb 0x7fc9e406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9e99e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9e99e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40114/0 (socket says 172.21.15.3:40114) 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.375+0000 7fc9e99e4640 1 -- 172.21.15.3:0/2141969132 learned_addr learned my addr 172.21.15.3:0/2141969132 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9ea1e5640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a5ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 -- 172.21.15.3:0/2141969132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 msgr2=0x562655a5ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a5ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 -- 172.21.15.3:0/2141969132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 msgr2=0x562655a5f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:45.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a5f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 -- 172.21.15.3:0/2141969132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562655a7ce60 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9e99e4640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009610 tx=0x7fc9e0097720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3240e6a7bd0135a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0020040 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc9e0097b90 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5626559c7b10 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9e009bc20 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9ee021740 10 client.15405 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562655adfc90 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.376+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5626559c7b10 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e00a3d00 con 0x562655a599d0 2024-09-12T02:18:45.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 10 client.15405 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 20 client.15405 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655ae4860 0x562655ae3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 4 client.15405 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "status"}) -- 0x5626559440e0 con 0x562655ae4860 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655ae4860 0x562655ae3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9e91e3640 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655ae4860 0x562655ae3570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76947d5a43134d8a server_cookie=32aefb4db12e0430 in_seq=0 out_seq=0 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9cd7fa640 10 client.15405 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9cd7fa640 1 -- 172.21.15.3:0/2141969132 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5626559440e0 con 0x562655ae4860 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9cd7fa640 10 client.15405 handle_command_reply: tid=0 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.377+0000 7fc9ee021740 1 client.15405 shutdown 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 msgr2=0x562655a5e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009610 tx=0x7fc9e0097720 comp rx=0 tx=0).stop 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 shutdown_connections 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562655a64bb0 0x562655a5f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x562655ae4860 0x562655ae3570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562655a5a2f0 0x562655a5ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 --2- 172.21.15.3:0/2141969132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562655a599d0 0x562655a5e7e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9cd7fa640 0 client.15405 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 >> 172.21.15.3:0/2141969132 conn(0x562655a55f60 msgr2=0x562655a55000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 shutdown_connections 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 1 -- 172.21.15.3:0/2141969132 wait complete. 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ee021740 20 client.15405 trim_cache size 0 max 16384 2024-09-12T02:18:45.383 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 4, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14619, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 125.641080524, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 29, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:45.384 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 149.06625111 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 msgr2=0x7fc9d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc04d000 tx=0x7fc9dc04a000 comp rx=0 tx=0).stop 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 msgr2=0x7fc9e41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.378+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d950 tx=0x7fc9e000de10 comp rx=0 tx=0).stop 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 shutdown_connections 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc9d00a2c10 0x7fc9d00a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e410c170 0x7fc9e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9e4103660 0x7fc9e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 --2- 172.21.15.3:0/1935320762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc9e4102d40 0x7fc9e41009a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 >> 172.21.15.3:0/1935320762 conn(0x7fc9e40fc7a0 msgr2=0x7fc9e40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 shutdown_connections 2024-09-12T02:18:45.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:45.379+0000 7fc9ebc48640 1 -- 172.21.15.3:0/1935320762 wait complete. 2024-09-12T02:18:45.968 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:18:46.224 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:46.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:46 smithi003 ceph-mon[31476]: pgmap v347: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-12T02:18:46.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:46 smithi026 ceph-mon[35496]: pgmap v347: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-12T02:18:46.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:46 smithi092 ceph-mon[36089]: pgmap v347: 97 pgs: 97 active+clean; 652 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 -- 172.21.15.3:0/3782834553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8103660 msgr2=0x7f0db810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/3782834553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8103660 0x7f0db810bb50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009900 tx=0x7f0dac02a480 comp rx=0 tx=0).stop 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 -- 172.21.15.3:0/3782834553 shutdown_connections 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/3782834553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db810c170 0x7f0db810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/3782834553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db8103660 0x7f0db810bb50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/3782834553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 0x7f0db8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 -- 172.21.15.3:0/3782834553 >> 172.21.15.3:0/3782834553 conn(0x7f0db80fc7a0 msgr2=0x7f0db80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 -- 172.21.15.3:0/3782834553 shutdown_connections 2024-09-12T02:18:46.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 -- 172.21.15.3:0/3782834553 wait complete. 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.694+0000 7f0dbd10e640 1 Processor -- start 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 -- start start 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 0x7f0db8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db8103660 0x7f0db8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111f70 con 0x7f0db810c170 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8111df0 con 0x7f0db8103660 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0dbd10e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81120f0 con 0x7f0db8102d40 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40122/0 (socket says 172.21.15.3:40122) 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 -- 172.21.15.3:0/4119756160 learned_addr learned my addr 172.21.15.3:0/4119756160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:46.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 0x7f0db8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db659c640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db8103660 0x7f0db8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 -- 172.21.15.3:0/4119756160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 msgr2=0x7f0db8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 0x7f0db8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 -- 172.21.15.3:0/4119756160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db8103660 msgr2=0x7f0db8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db8103660 0x7f0db8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.695+0000 7f0db759e640 1 -- 172.21.15.3:0/4119756160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db819d400 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0db759e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0db000eaa0 tx=0x7f0db000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0d9ffff640 1 -- 172.21.15.3:0/4119756160 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db000cdb0 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db819d6f0 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0d9ffff640 1 -- 172.21.15.3:0/4119756160 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db0005020 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0d9ffff640 1 -- 172.21.15.3:0/4119756160 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00116b0 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.696+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81a4fd0 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.698+0000 7f0d9ffff640 1 -- 172.21.15.3:0/4119756160 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0db0005560 con 0x7f0db810c170 2024-09-12T02:18:46.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 Processor -- start 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 -- start start 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0d9ffff640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 0x7f0da00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1ab95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 0x55eb1aba2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aafc090 con 0x55eb1ab962f0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0d9ffff640 1 -- 172.21.15.3:0/4119756160 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0db00cf380 con 0x7f0db810c170 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aaa3a70 con 0x55eb1ab959d0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aaf8050 con 0x55eb1aba0bb0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db659c640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 0x7f0da00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1ab95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40126/0 (socket says 172.21.15.3:40126) 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1ab95db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:53896/0 (socket says 172.21.15.3:53896) 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 0x55eb1aba2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.699+0000 7f0db659c640 1 -- 172.21.15.3:0/573770230 learned_addr learned my addr 172.21.15.3:0/573770230 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db659c640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 0x7f0da00a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0096c0 tx=0x7f0dac031800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 -- 172.21.15.3:0/573770230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 msgr2=0x55eb1ab95db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1ab95db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 -- 172.21.15.3:0/573770230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 msgr2=0x55eb1aba0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 -- 172.21.15.3:0/573770230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1aab0130 con 0x55eb1aba0bb0 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db659c640 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0db759e640 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 0x55eb1aba2fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0db00cccc0 tx=0x7f0db009fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614b9c3950cdff85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0d9d7fa640 1 -- 172.21.15.3:0/573770230 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db001f070 con 0x55eb1aba0bb0 2024-09-12T02:18:46.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0d9d7fa640 1 -- 172.21.15.3:0/573770230 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db00ccf10 con 0x55eb1aba0bb0 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.700+0000 7f0d9d7fa640 1 -- 172.21.15.3:0/573770230 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db009a980 con 0x55eb1aba0bb0 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/573770230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 msgr2=0x55eb1aba2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 0x55eb1aba2fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0db00cccc0 tx=0x7f0db009fdc0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/573770230 shutdown_connections 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1aba0bb0 0x55eb1aba2fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1aba0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/573770230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1ab95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/573770230 >> 172.21.15.3:0/573770230 conn(0x55eb1ab91f60 msgr2=0x55eb1ab92340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/573770230 shutdown_connections 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/573770230 wait complete. 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- start start 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 Processor -- start 2024-09-12T02:18:46.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1aba03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 0x55eb1ab9d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aba7b90 con 0x55eb1aba0bb0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aab6f20 con 0x55eb1ab959d0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aba7d10 con 0x55eb1ab962f0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.701+0000 7f0dbf4e7740 10 client.0 _ll_register_callbacks cb 0x55eb1aa80030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 0x55eb1ab9d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1aba03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 0x55eb1ab9d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48314/0 (socket says 172.21.15.3:48314) 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40140/0 (socket says 172.21.15.3:40140) 2024-09-12T02:18:46.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db659c640 1 -- 172.21.15.3:0/1465011483 learned_addr learned my addr 172.21.15.3:0/1465011483 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 -- 172.21.15.3:0/1465011483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 msgr2=0x55eb1ab9d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 0x55eb1ab9d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 -- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 msgr2=0x55eb1aba03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1aba03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 -- 172.21.15.3:0/1465011483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1a86d2b0 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0db759e640 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55eb1ab96990 tx=0x7f0db0097c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6cf797c63264a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0d9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00b9070 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0db009b0f0 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db00ccf10 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0db001f070 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0dbf4e7740 10 client.15417 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.702+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb1abb3440 con 0x55eb1aba0bb0 2024-09-12T02:18:46.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db00ccf10 con 0x55eb1aba0bb0 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0db000f040 con 0x55eb1aba0bb0 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 10 client.15417 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 10 client.15417 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 20 client.15417 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1abb4790 0x55eb1abb6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 4 client.15417 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1ab09600 con 0x55eb1abb4790 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.703+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1abb4790 0x55eb1abb6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.704+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1abb4790 0x55eb1abb6910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b3213b59c904292 server_cookie=30f0e6edebcf404c in_seq=0 out_seq=0 2024-09-12T02:18:46.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.704+0000 7f0d9aff5640 10 client.15417 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.704+0000 7f0d9aff5640 1 -- 172.21.15.3:0/1465011483 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb1ab09600 con 0x55eb1abb4790 2024-09-12T02:18:46.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.704+0000 7f0d9aff5640 10 client.15417 handle_command_reply: tid=0 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.704+0000 7f0dbf4e7740 1 client.15417 shutdown 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 msgr2=0x55eb1ab9dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55eb1ab96990 tx=0x7f0db0097c50 comp rx=0 tx=0).stop 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 shutdown_connections 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9dc70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab962f0 0x55eb1ab9d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1abb4790 0x55eb1abb6910 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1465011483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1ab959d0 0x55eb1aba03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0d9aff5640 0 client.15417 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 >> 172.21.15.3:0/1465011483 conn(0x55eb1ab91f60 msgr2=0x55eb1ab97c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 shutdown_connections 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1465011483 wait complete. 2024-09-12T02:18:46.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.705+0000 7f0dbf4e7740 20 client.15417 trim_cache size 0 max 16384 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 Processor -- start 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 -- start start 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abb6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1abb4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb4c50 con 0x55eb1ab962f0 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb5320 con 0x55eb1aba0bb0 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb59f0 con 0x55eb1ab959d0 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40154/0 (socket says 172.21.15.3:40154) 2024-09-12T02:18:46.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 -- 172.21.15.3:0/1833117454 learned_addr learned my addr 172.21.15.3:0/1833117454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abb6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db759e640 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1abb4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 -- 172.21.15.3:0/1833117454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 msgr2=0x55eb1abb6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abb6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 -- 172.21.15.3:0/1833117454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 msgr2=0x55eb1abb4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1abb4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 -- 172.21.15.3:0/1833117454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1aba7ca0 con 0x55eb1ab962f0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.723+0000 7f0db659c640 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009230 tx=0x7f0dac047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de474d013d077b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0d9cff9640 1 -- 172.21.15.3:0/1833117454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac035040 con 0x55eb1ab962f0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0d9cff9640 1 -- 172.21.15.3:0/1833117454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dac0044c0 con 0x55eb1ab962f0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0d9cff9640 1 -- 172.21.15.3:0/1833117454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac03d3f0 con 0x55eb1ab962f0 2024-09-12T02:18:46.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1833117454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 msgr2=0x55eb1abb5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009230 tx=0x7f0dac047000 comp rx=0 tx=0).stop 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1833117454 shutdown_connections 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1abb4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abb5b70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/1833117454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abb6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1833117454 >> 172.21.15.3:0/1833117454 conn(0x55eb1ab91f60 msgr2=0x55eb1ab985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1833117454 shutdown_connections 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.724+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/1833117454 wait complete. 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 -- start start 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 Processor -- start 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abbee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb6790 con 0x55eb1ab962f0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb60c0 con 0x55eb1aba0bb0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb1abb6e60 con 0x55eb1ab959d0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0dbf4e7740 10 client.0 _ll_register_callbacks cb 0x7f0dac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40160/0 (socket says 172.21.15.3:40160) 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 -- 172.21.15.3:0/2543035191 learned_addr learned my addr 172.21.15.3:0/2543035191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abbee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db759e640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 -- 172.21.15.3:0/2543035191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 msgr2=0x55eb1abbee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abbee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.725+0000 7f0db659c640 1 -- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 msgr2=0x55eb1ab9a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:46.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0db659c640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0db659c640 1 -- 172.21.15.3:0/2543035191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1ab9a8f0 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0db759e640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0db659c640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005c40 tx=0x7f0dac02ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141e363146775519 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0d9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003fd0 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0dac04a440 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb1ab03b10 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0dac04a640 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0dbf4e7740 10 client.15423 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb1abbe750 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.726+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eb1ab03b10 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0dac054430 con 0x55eb1ab962f0 2024-09-12T02:18:46.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 10 client.15423 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:18:46.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 20 client.15423 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1ac23e40 0x55eb1ac22b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:47.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 4 client.15423 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "status"}) -- 0x55eb1aa800e0 con 0x55eb1ac23e40 2024-09-12T02:18:47.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1ac23e40 0x55eb1ac22b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0d9aff5640 10 client.15423 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0db6d9d640 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1ac23e40 0x55eb1ac22b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59a5140ee2fcdbe3 server_cookie=f1303c43e6cc0131 in_seq=0 out_seq=0 2024-09-12T02:18:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0d9aff5640 1 -- 172.21.15.3:0/2543035191 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55eb1aa800e0 con 0x55eb1ac23e40 2024-09-12T02:18:47.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0d9aff5640 10 client.15423 handle_command_reply: tid=0 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.727+0000 7f0dbf4e7740 1 client.15423 shutdown 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 msgr2=0x55eb1abbf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0dac005c40 tx=0x7f0dac02ac00 comp rx=0 tx=0).stop 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 shutdown_connections 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55eb1ac23e40 0x55eb1ac22b50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb1aba0bb0 0x55eb1ab9a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb1ab962f0 0x55eb1abbf370 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 --2- 172.21.15.3:0/2543035191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb1ab959d0 0x55eb1abbee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 >> 172.21.15.3:0/2543035191 conn(0x55eb1ab91f60 msgr2=0x55eb1ab98de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0d9aff5640 0 client.15423 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 shutdown_connections 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 1 -- 172.21.15.3:0/2543035191 wait complete. 2024-09-12T02:18:47.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbf4e7740 20 client.15423 trim_cache size 0 max 16384 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 3, 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24482, 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 127.566561443, 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 29, 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:47.179 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:47.180 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 147.54869653 2024-09-12T02:18:47.180 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 msgr2=0x7f0da00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:47.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 0x7f0da00a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0096c0 tx=0x7f0dac031800 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 msgr2=0x7f0db819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.728+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0db000eaa0 tx=0x7f0db000ef60 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 shutdown_connections 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0da00a2c10 0x7f0da00a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0db810c170 0x7f0db819cd20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0db8103660 0x7f0db8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 --2- 172.21.15.3:0/4119756160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0db8102d40 0x7f0db8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 >> 172.21.15.3:0/4119756160 conn(0x7f0db80fc7a0 msgr2=0x7f0db80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 shutdown_connections 2024-09-12T02:18:47.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:46.729+0000 7f0dbd10e640 1 -- 172.21.15.3:0/4119756160 wait complete. 2024-09-12T02:18:47.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:18:47.737 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 -- 172.21.15.3:0/1262435481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6960102d40 msgr2=0x7f6960103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 --2- 172.21.15.3:0/1262435481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6960102d40 0x7f6960103120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f695002a480 comp rx=0 tx=0).stop 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 -- 172.21.15.3:0/1262435481 shutdown_connections 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 --2- 172.21.15.3:0/1262435481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f696010c170 0x7f696010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 --2- 172.21.15.3:0/1262435481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f696010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 --2- 172.21.15.3:0/1262435481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6960102d40 0x7f6960103120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 -- 172.21.15.3:0/1262435481 >> 172.21.15.3:0/1262435481 conn(0x7f69600fc7a0 msgr2=0x7f69600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 -- 172.21.15.3:0/1262435481 shutdown_connections 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.228+0000 7f6966154640 1 -- 172.21.15.3:0/1262435481 wait complete. 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 Processor -- start 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 -- start start 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6960102d40 0x7f69601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f69601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111fd0 con 0x7f696010c170 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111e50 con 0x7f6960102d40 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f6966154640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6960112150 con 0x7f6960103660 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f695ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f695ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40180/0 (socket says 172.21.15.3:40180) 2024-09-12T02:18:48.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f695ffff640 1 -- 172.21.15.3:0/3994826628 learned_addr learned my addr 172.21.15.3:0/3994826628 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.229+0000 7f695f7fe640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6960102d40 0x7f69601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695effd640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f69601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 -- 172.21.15.3:0/3994826628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 msgr2=0x7f69601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f69601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 -- 172.21.15.3:0/3994826628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6960102d40 msgr2=0x7f69601984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6960102d40 0x7f69601984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 -- 172.21.15.3:0/3994826628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696019d460 con 0x7f696010c170 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695effd640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f69601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695ffff640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f695800b970 tx=0x7f695800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695cff9640 1 -- 172.21.15.3:0/3994826628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958019070 con 0x7f696010c170 2024-09-12T02:18:48.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695cff9640 1 -- 172.21.15.3:0/3994826628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695800cb40 con 0x7f696010c170 2024-09-12T02:18:48.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696019d750 con 0x7f696010c170 2024-09-12T02:18:48.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f695cff9640 1 -- 172.21.15.3:0/3994826628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958007650 con 0x7f696010c170 2024-09-12T02:18:48.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.230+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a5030 con 0x7f696010c170 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.232+0000 7f695cff9640 1 -- 172.21.15.3:0/3994826628 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f695801c070 con 0x7f696010c170 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 Processor -- start 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 -- start start 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b4eed9090 con 0x564b4ef729d0 2024-09-12T02:18:48.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b4ee80a70 con 0x564b4ef7dbb0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f696852d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b4eed5050 con 0x564b4ef732f0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40186/0 (socket says 172.21.15.3:40186) 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 -- 172.21.15.3:0/2000509485 learned_addr learned my addr 172.21.15.3:0/2000509485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695f7fe640 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695cff9640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 -- 172.21.15.3:0/2000509485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 msgr2=0x564b4ef7d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 -- 172.21.15.3:0/2000509485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 msgr2=0x564b4ef7ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.233+0000 7f695effd640 1 -- 172.21.15.3:0/2000509485 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4ee8d130 con 0x564b4ef729d0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f695ffff640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f695f7fe640 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f695cff9640 1 -- 172.21.15.3:0/3994826628 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f69580ce6c0 con 0x7f696010c170 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f695effd640 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f695400bd40 tx=0x7f6954011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a991e86f2e20b836 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f6945ffb640 1 -- 172.21.15.3:0/2000509485 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954016070 con 0x564b4ef729d0 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f6945ffb640 1 -- 172.21.15.3:0/2000509485 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695401b940 con 0x564b4ef729d0 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.234+0000 7f6945ffb640 1 -- 172.21.15.3:0/2000509485 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954012970 con 0x564b4ef729d0 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- 172.21.15.3:0/2000509485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 msgr2=0x564b4ef72db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f695400bd40 tx=0x7f6954011750 comp rx=0 tx=0).stop 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f695ffff640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 0x7f69480a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f695809af00 tx=0x7f6958091040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- 172.21.15.3:0/2000509485 shutdown_connections 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 --2- 172.21.15.3:0/2000509485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef72db0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- 172.21.15.3:0/2000509485 >> 172.21.15.3:0/2000509485 conn(0x564b4ef6ef60 msgr2=0x564b4ef6f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- 172.21.15.3:0/2000509485 shutdown_connections 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- 172.21.15.3:0/2000509485 wait complete. 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 -- start start 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.235+0000 7f696852d740 1 Processor -- start 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef84b90 con 0x564b4ef729d0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b4ee93f20 con 0x564b4ef7dbb0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef84d10 con 0x564b4ef732f0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f696852d740 10 client.0 _ll_register_callbacks cb 0x564b4ee5d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40198/0 (socket says 172.21.15.3:40198) 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 -- 172.21.15.3:0/835737255 learned_addr learned my addr 172.21.15.3:0/835737255 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695f7fe640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 -- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 msgr2=0x564b4ef7a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 -- 172.21.15.3:0/835737255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 msgr2=0x564b4ef7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 -- 172.21.15.3:0/835737255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4ec4a2b0 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f695effd640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6954011c50 tx=0x7f69540125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c2875a75802049 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.236+0000 7f693effd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954016070 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f695401dda0 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695401b940 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f695401baf0 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b4ef90500 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695401b940 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6954012a10 con 0x564b4ef729d0 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 10 client.15435 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 20 client.15435 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:48.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4ef926c0 0x564b4ef93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 4 client.15435 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:48.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.237+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b4eee6600 con 0x564b4ef926c0 2024-09-12T02:18:48.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.238+0000 7f695f7fe640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4ef926c0 0x564b4ef93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.238+0000 7f695f7fe640 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4ef926c0 0x564b4ef93b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7087f2831866de8a server_cookie=58799640cb0a1324 in_seq=0 out_seq=0 2024-09-12T02:18:48.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.238+0000 7f693effd640 10 client.15435 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f693effd640 1 -- 172.21.15.3:0/835737255 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b4eee6600 con 0x564b4ef926c0 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f693effd640 10 client.15435 handle_command_reply: tid=0 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 client.15435 shutdown 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 msgr2=0x564b4ef7d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6954011c50 tx=0x7f69540125d0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 shutdown_connections 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4ef926c0 0x564b4ef93b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef7a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 --2- 172.21.15.3:0/835737255 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef7d490 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 >> 172.21.15.3:0/835737255 conn(0x564b4ef6ef60 msgr2=0x564b4ef74c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f693effd640 0 client.15435 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 shutdown_connections 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 1 -- 172.21.15.3:0/835737255 wait complete. 2024-09-12T02:18:48.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.239+0000 7f696852d740 20 client.15435 trim_cache size 0 max 16384 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 Processor -- start 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 -- start start 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef729d0 0x564b4ef7c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 0x564b4ef77fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef91e70 con 0x564b4ef732f0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef92540 con 0x564b4ef729d0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f696852d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef92c10 con 0x564b4ef7dbb0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40200/0 (socket says 172.21.15.3:40200) 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f695f7fe640 1 -- 172.21.15.3:0/25336999 learned_addr learned my addr 172.21.15.3:0/25336999 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.257+0000 7f695effd640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef729d0 0x564b4ef7c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695effd640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 0x564b4ef77fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 -- 172.21.15.3:0/25336999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 msgr2=0x564b4ef77fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 0x564b4ef77fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 -- 172.21.15.3:0/25336999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef729d0 msgr2=0x564b4ef7c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef729d0 0x564b4ef7c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 -- 172.21.15.3:0/25336999 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4ef84ca0 con 0x564b4ef732f0 2024-09-12T02:18:48.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695effd640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 0x564b4ef77fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:48.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f695f7fe640 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f6950031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07feb09ba43339a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f69457fa640 1 -- 172.21.15.3:0/25336999 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950035040 con 0x564b4ef732f0 2024-09-12T02:18:48.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f69457fa640 1 -- 172.21.15.3:0/25336999 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6950031a70 con 0x564b4ef732f0 2024-09-12T02:18:48.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.258+0000 7f69457fa640 1 -- 172.21.15.3:0/25336999 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950004c60 con 0x564b4ef732f0 2024-09-12T02:18:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- 172.21.15.3:0/25336999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 msgr2=0x564b4ef77a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f6950031980 comp rx=0 tx=0).stop 2024-09-12T02:18:48.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- 172.21.15.3:0/25336999 shutdown_connections 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef7dbb0 0x564b4ef77fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef732f0 0x564b4ef77a80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 --2- 172.21.15.3:0/25336999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef729d0 0x564b4ef7c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- 172.21.15.3:0/25336999 >> 172.21.15.3:0/25336999 conn(0x564b4ef6ef60 msgr2=0x564b4ef7bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- 172.21.15.3:0/25336999 shutdown_connections 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- 172.21.15.3:0/25336999 wait complete. 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 -- start start 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.259+0000 7f696852d740 1 Processor -- start 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef97630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef97b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef939b0 con 0x564b4ef729d0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef932e0 con 0x564b4ef7dbb0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564b4ef94080 con 0x564b4ef732f0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 10 client.0 _ll_register_callbacks cb 0x7f695400bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40214/0 (socket says 172.21.15.3:40214) 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 -- 172.21.15.3:0/3204909615 learned_addr learned my addr 172.21.15.3:0/3204909615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695f7fe640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef97630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef97b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 -- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 msgr2=0x564b4ef97630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef97630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 -- 172.21.15.3:0/3204909615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 msgr2=0x564b4ef97b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:48.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef97b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 -- 172.21.15.3:0/3204909615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4ef98130 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f695effd640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695400b900 tx=0x7f695402d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92a882309f8b25d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f693effd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954016070 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6954012640 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.260+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6950031a70 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6954011c80 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 10 client.15441 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564b4effcb10 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6950031a70 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f695401d740 con 0x564b4ef729d0 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 10 client.15441 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:48.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 10 client.15441 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 20 client.15441 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4f001660 0x564b4f000370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 4 client.15441 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "status"}) -- 0x564b4eee0b10 con 0x564b4f001660 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.261+0000 7f695f7fe640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4f001660 0x564b4f000370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f695f7fe640 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4f001660 0x564b4f000370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5818221b506f4f69 server_cookie=fcc488c422cfebb7 in_seq=0 out_seq=0 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f693effd640 10 client.15441 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f693effd640 1 -- 172.21.15.3:0/3204909615 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564b4eee0b10 con 0x564b4f001660 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f693effd640 10 client.15441 handle_command_reply: tid=0 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 client.15441 shutdown 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 msgr2=0x564b4ef9c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f695400b900 tx=0x7f695402d040 comp rx=0 tx=0).stop 2024-09-12T02:18:48.267 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 shutdown_connections 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b4ef7dbb0 0x564b4ef97b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x564b4f001660 0x564b4f000370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564b4ef732f0 0x564b4ef97630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 --2- 172.21.15.3:0/3204909615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564b4ef729d0 0x564b4ef9c300 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 >> 172.21.15.3:0/3204909615 conn(0x564b4ef6ef60 msgr2=0x564b4ef6e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f693effd640 0 client.15441 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 shutdown_connections 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.262+0000 7f696852d740 1 -- 172.21.15.3:0/3204909615 wait complete. 2024-09-12T02:18:48.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f696852d740 20 client.15441 trim_cache size 0 max 16384 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 2, 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24487, 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 130.11349348499999, 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 29, 2024-09-12T02:18:48.269 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 166.088733961 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 msgr2=0x7f69480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 0x7f69480a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f695809af00 tx=0x7f6958091040 comp rx=0 tx=0).stop 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 msgr2=0x7f696019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:48.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f695800b970 tx=0x7f695800be30 comp rx=0 tx=0).stop 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 shutdown_connections 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f69480a2c10 0x7f69480a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f696010c170 0x7f696019cd80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6960103660 0x7f69601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 --2- 172.21.15.3:0/3994826628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6960102d40 0x7f69601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 >> 172.21.15.3:0/3994826628 conn(0x7f69600fc7a0 msgr2=0x7f69600fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 shutdown_connections 2024-09-12T02:18:48.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:48.263+0000 7f6966154640 1 -- 172.21.15.3:0/3994826628 wait complete. 2024-09-12T02:18:48.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:48 smithi026 ceph-mon[35496]: pgmap v348: 97 pgs: 97 active+clean; 662 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-12T02:18:48.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:48 smithi003 ceph-mon[31476]: pgmap v348: 97 pgs: 97 active+clean; 662 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-12T02:18:48.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:48 smithi092 ceph-mon[36089]: pgmap v348: 97 pgs: 97 active+clean; 662 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-12T02:18:48.947 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:18:49.200 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 -- 172.21.15.3:0/4126100873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60106f20 msgr2=0x7fdd6010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 --2- 172.21.15.3:0/4126100873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60106f20 0x7fdd6010eaa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009ca0 tx=0x7fdd5802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 -- 172.21.15.3:0/4126100873 shutdown_connections 2024-09-12T02:18:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 --2- 172.21.15.3:0/4126100873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60106f20 0x7fdd6010eaa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 --2- 172.21.15.3:0/4126100873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd601027a0 0x7fdd601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 --2- 172.21.15.3:0/4126100873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd60101df0 0x7fdd601021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 -- 172.21.15.3:0/4126100873 >> 172.21.15.3:0/4126100873 conn(0x7fdd600fd8b0 msgr2=0x7fdd600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 -- 172.21.15.3:0/4126100873 shutdown_connections 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.622+0000 7fdd65418640 1 -- 172.21.15.3:0/4126100873 wait complete. 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.623+0000 7fdd65418640 1 Processor -- start 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.623+0000 7fdd65418640 1 -- start start 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd601027a0 0x7fdd60198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd60106f20 0x7fdd6019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60111f80 con 0x7fdd60101df0 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60111e00 con 0x7fdd60106f20 2024-09-12T02:18:49.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd65418640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdd60112100 con 0x7fdd601027a0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45090/0 (socket says 172.21.15.3:45090) 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 -- 172.21.15.3:0/1138062366 learned_addr learned my addr 172.21.15.3:0/1138062366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd601027a0 0x7fdd60198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd60106f20 0x7fdd6019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 -- 172.21.15.3:0/1138062366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd601027a0 msgr2=0x7fdd60198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd601027a0 0x7fdd60198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 -- 172.21.15.3:0/1138062366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd60106f20 msgr2=0x7fdd6019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd60106f20 0x7fdd6019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 -- 172.21.15.3:0/1138062366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6019d3d0 con 0x7fdd60101df0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd5effd640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d770 tx=0x7fdd5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd47fff640 1 -- 172.21.15.3:0/1138062366 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50017070 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.624+0000 7fdd47fff640 1 -- 172.21.15.3:0/1138062366 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd50005500 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.625+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6019d6c0 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.625+0000 7fdd47fff640 1 -- 172.21.15.3:0/1138062366 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004e00 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.625+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601a4fa0 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd47fff640 1 -- 172.21.15.3:0/1138062366 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fdd50004fa0 con 0x7fdd60101df0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 Processor -- start 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 -- start start 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb9275db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb9280670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561cb91dc090 con 0x561cb9280bb0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cb9183a70 con 0x561cb92759d0 2024-09-12T02:18:49.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd677f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cb91d8050 con 0x561cb92762f0 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd5e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd5e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45092/0 (socket says 172.21.15.3:45092) 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd5e7fc640 1 -- 172.21.15.3:0/267771880 learned_addr learned my addr 172.21.15.3:0/267771880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd5effd640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb9280670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb9275db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.628+0000 7fdd47fff640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 0x7fdd480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 -- 172.21.15.3:0/267771880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 msgr2=0x561cb9280670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb9280670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 -- 172.21.15.3:0/267771880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 msgr2=0x561cb9275db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb9275db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 -- 172.21.15.3:0/267771880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb9190130 con 0x561cb9280bb0 2024-09-12T02:18:49.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 0x7fdd480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5effd640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb9280670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd47fff640 1 -- 172.21.15.3:0/1138062366 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdd500cec60 con 0x7fdd60101df0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400ab70 tx=0x7fdd5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b366641940f6dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd457fa640 1 -- 172.21.15.3:0/267771880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5401d070 con 0x561cb9280bb0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 0x7fdd480a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009a60 tx=0x7fdd5802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd457fa640 1 -- 172.21.15.3:0/267771880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54018b20 con 0x561cb9280bb0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.629+0000 7fdd457fa640 1 -- 172.21.15.3:0/267771880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54010a90 con 0x561cb9280bb0 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- 172.21.15.3:0/267771880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 msgr2=0x561cb9282fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400ab70 tx=0x7fdd5400f790 comp rx=0 tx=0).stop 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- 172.21.15.3:0/267771880 shutdown_connections 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9282fa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb9280670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 --2- 172.21.15.3:0/267771880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb9275db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- 172.21.15.3:0/267771880 >> 172.21.15.3:0/267771880 conn(0x561cb9271f60 msgr2=0x561cb9272340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- 172.21.15.3:0/267771880 shutdown_connections 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- 172.21.15.3:0/267771880 wait complete. 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 -- start start 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.630+0000 7fdd677f1740 1 Processor -- start 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb927d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb9280bb0 0x561cb927dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561cb9283780 con 0x561cb92759d0 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cb9196f20 con 0x561cb92762f0 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cb9283900 con 0x561cb9280bb0 2024-09-12T02:18:49.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd677f1740 10 client.0 _ll_register_callbacks cb 0x561cb9160030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45094/0 (socket says 172.21.15.3:45094) 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 -- 172.21.15.3:0/2803586214 learned_addr learned my addr 172.21.15.3:0/2803586214 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb9280bb0 0x561cb927dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb927d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 -- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb9280bb0 msgr2=0x561cb927dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb9280bb0 0x561cb927dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 -- 172.21.15.3:0/2803586214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 msgr2=0x561cb927d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb927d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 -- 172.21.15.3:0/2803586214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb8f4d2b0 con 0x561cb92759d0 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb927d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:49.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdd50097d20 tx=0x7fdd5009acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212390d1476374b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.631+0000 7fdd42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50020070 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdd50009cc0 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd54018b20 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd5009ba50 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cb928f0f0 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54018b20 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd500b4120 con 0x561cb92759d0 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 10 client.15453 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 20 client.15453 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:49.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb9291720 0x561cb9292b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 4 client.15453 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.632+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb91e9600 con 0x561cb9291720 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.633+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb9291720 0x561cb9292b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.633+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb9291720 0x561cb9292b90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32569cf35f5aafd2 server_cookie=1d7d545ba865d67f in_seq=0 out_seq=0 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.633+0000 7fdd42ff5640 10 client.15453 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2803586214 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cb91e9600 con 0x561cb9291720 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd42ff5640 10 client.15453 handle_command_reply: tid=0 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 client.15453 shutdown 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 msgr2=0x561cb9280440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdd50097d20 tx=0x7fdd5009acb0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 shutdown_connections 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb9291720 0x561cb9292b90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb9280bb0 0x561cb927dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb927d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd42ff5640 0 client.15453 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2803586214 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb92759d0 0x561cb9280440 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 >> 172.21.15.3:0/2803586214 conn(0x561cb9271f60 msgr2=0x561cb9277c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 shutdown_connections 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 1 -- 172.21.15.3:0/2803586214 wait complete. 2024-09-12T02:18:49.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.634+0000 7fdd677f1740 20 client.15453 trim_cache size 0 max 16384 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 Processor -- start 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 -- start start 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb927d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb927dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561cb9290ed0 con 0x561cb9280bb0 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cb92915a0 con 0x561cb92759d0 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd677f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cb9291c70 con 0x561cb92762f0 2024-09-12T02:18:49.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45100/0 (socket says 172.21.15.3:45100) 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.652+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2141881822 learned_addr learned my addr 172.21.15.3:0/2141881822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb927d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb927dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2141881822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 msgr2=0x561cb927dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb927dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2141881822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 msgr2=0x561cb927d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb927d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2141881822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb92730d0 con 0x561cb9280bb0 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb927dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdd58035000 tx=0x7fdd58042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1aaecd5d2c0a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd44ff9640 1 -- 172.21.15.3:0/2141881822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58043070 con 0x561cb9280bb0 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd44ff9640 1 -- 172.21.15.3:0/2141881822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5804f030 con 0x561cb9280bb0 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.653+0000 7fdd44ff9640 1 -- 172.21.15.3:0/2141881822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5802f610 con 0x561cb9280bb0 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- 172.21.15.3:0/2141881822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 msgr2=0x561cb927c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdd58035000 tx=0x7fdd58042000 comp rx=0 tx=0).stop 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- 172.21.15.3:0/2141881822 shutdown_connections 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb927c060 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92762f0 0x561cb927dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2141881822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92759d0 0x561cb927d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- 172.21.15.3:0/2141881822 >> 172.21.15.3:0/2141881822 conn(0x561cb9271f60 msgr2=0x561cb927ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- 172.21.15.3:0/2141881822 shutdown_connections 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- 172.21.15.3:0/2141881822 wait complete. 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 -- start start 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.654+0000 7fdd677f1740 1 Processor -- start 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92759d0 0x561cb9295dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb9296310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x561cb9292a10 con 0x561cb9280bb0 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x561cb9292340 con 0x561cb92762f0 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561cb92930e0 con 0x561cb92759d0 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd677f1740 10 client.0 _ll_register_callbacks cb 0x7fdd500056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45106/0 (socket says 172.21.15.3:45106) 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2812868495 learned_addr learned my addr 172.21.15.3:0/2812868495 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5e7fc640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb9296310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92759d0 0x561cb9295dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92759d0 msgr2=0x561cb9295dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92759d0 0x561cb9295dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2812868495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 msgr2=0x561cb9296310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb9296310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 -- 172.21.15.3:0/2812868495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb92fa040 con 0x561cb9280bb0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd5f7fe640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdd580044f0 tx=0x7fdd580040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a02a26a7de020ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.655+0000 7fdd42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5805b020 con 0x561cb9280bb0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdd58031650 con 0x561cb9280bb0 2024-09-12T02:18:49.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd5804f030 con 0x561cb9280bb0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd58066050 con 0x561cb9280bb0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 10 client.15459 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561cb92fb320 con 0x561cb9280bb0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5804f030 con 0x561cb9280bb0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdd58046070 con 0x561cb9280bb0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 10 client.15459 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 20 client.15459 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb92ffa20 0x561cb92fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 4 client.15459 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.656+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "status"}) -- 0x561cb91e3b10 con 0x561cb92ffa20 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb92ffa20 0x561cb92fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:49.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd5effd640 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb92ffa20 0x561cb92fe730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfba43d9dc939095 server_cookie=985d7f86e34b59db in_seq=0 out_seq=0 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd42ff5640 10 client.15459 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd42ff5640 1 -- 172.21.15.3:0/2812868495 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561cb91e3b10 con 0x561cb92ffa20 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd42ff5640 10 client.15459 handle_command_reply: tid=0 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 client.15459 shutdown 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 msgr2=0x561cb9296850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdd580044f0 tx=0x7fdd580040e0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 shutdown_connections 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x561cb92ffa20 0x561cb92fe730 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561cb9280bb0 0x561cb9296850 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561cb92762f0 0x561cb9296310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 --2- 172.21.15.3:0/2812868495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561cb92759d0 0x561cb9295dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 >> 172.21.15.3:0/2812868495 conn(0x561cb9271f60 msgr2=0x561cb9272340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.657+0000 7fdd42ff5640 0 client.15459 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 shutdown_connections 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd677f1740 1 -- 172.21.15.3:0/2812868495 wait complete. 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd677f1740 20 client.15459 trim_cache size 0 max 16384 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 1, 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:49.666 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 132.51597587500001, 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 29, 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 164.743408824 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:49.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 msgr2=0x7fdd480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 0x7fdd480a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009a60 tx=0x7fdd5802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 msgr2=0x7fdd60198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d770 tx=0x7fdd5000dc30 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 shutdown_connections 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdd480a2c10 0x7fdd480a50d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd60106f20 0x7fdd6019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd601027a0 0x7fdd60198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 --2- 172.21.15.3:0/1138062366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd60101df0 0x7fdd60198420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.658+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 >> 172.21.15.3:0/1138062366 conn(0x7fdd600fd8b0 msgr2=0x7fdd600fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.659+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 shutdown_connections 2024-09-12T02:18:49.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:49.659+0000 7fdd65418640 1 -- 172.21.15.3:0/1138062366 wait complete. 2024-09-12T02:18:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:50 smithi003 ceph-mon[31476]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:50.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:18:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:50 smithi026 ceph-mon[35496]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:50.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:50 smithi092 ceph-mon[36089]: pgmap v349: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:50.658 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 -- 172.21.15.3:0/1453938345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa614103760 msgr2=0x7fa614107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 --2- 172.21.15.3:0/1453938345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa614103760 0x7fa614107ba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa608009930 tx=0x7fa60802a420 comp rx=0 tx=0).stop 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 -- 172.21.15.3:0/1453938345 shutdown_connections 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 --2- 172.21.15.3:0/1453938345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6141082e0 0x7fa61410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 --2- 172.21.15.3:0/1453938345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa614103760 0x7fa614107ba0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 --2- 172.21.15.3:0/1453938345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 0x7fa614103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 -- 172.21.15.3:0/1453938345 >> 172.21.15.3:0/1453938345 conn(0x7fa6140fc7d0 msgr2=0x7fa6140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 -- 172.21.15.3:0/1453938345 shutdown_connections 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 -- 172.21.15.3:0/1453938345 wait complete. 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.150+0000 7fa6195bb640 1 Processor -- start 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 -- start start 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 0x7fa614198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa614103760 0x7fa614198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112070 con 0x7fa6141082e0 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa614111ef0 con 0x7fa614102db0 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6195bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa6141121f0 con 0x7fa614103760 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45126/0 (socket says 172.21.15.3:45126) 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 -- 172.21.15.3:0/2296487137 learned_addr learned my addr 172.21.15.3:0/2296487137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa612ffd640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 0x7fa614198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6127fc640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa614103760 0x7fa614198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 -- 172.21.15.3:0/2296487137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa614103760 msgr2=0x7fa614198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa614103760 0x7fa614198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 -- 172.21.15.3:0/2296487137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 msgr2=0x7fa614198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 0x7fa614198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.151+0000 7fa6137fe640 1 -- 172.21.15.3:0/2296487137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61419d540 con 0x7fa6141082e0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa6137fe640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00eaa0 tx=0x7fa60c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa5fbfff640 1 -- 172.21.15.3:0/2296487137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00cdb0 con 0x7fa6141082e0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa5fbfff640 1 -- 172.21.15.3:0/2296487137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60c005020 con 0x7fa6141082e0 2024-09-12T02:18:51.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa5fbfff640 1 -- 172.21.15.3:0/2296487137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0116b0 con 0x7fa6141082e0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419d830 con 0x7fa6141082e0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.152+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6141a5110 con 0x7fa6141082e0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.154+0000 7fa5fbfff640 1 -- 172.21.15.3:0/2296487137 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa60c005560 con 0x7fa6141082e0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.154+0000 7fa61b994740 1 Processor -- start 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.154+0000 7fa61b994740 1 -- start start 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.154+0000 7fa61b994740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa61b994740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa61b994740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa61b994740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595a063e090 con 0x5595a06d79d0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa61b994740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595a05e5a70 con 0x5595a06e2bb0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa61b994740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595a063a050 con 0x5595a06d82f0 2024-09-12T02:18:51.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45138/0 (socket says 172.21.15.3:45138) 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 -- 172.21.15.3:0/2815149048 learned_addr learned my addr 172.21.15.3:0/2815149048 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa612ffd640 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa5fbfff640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 0x7fa5fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 -- 172.21.15.3:0/2815149048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 msgr2=0x5595a06e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 -- 172.21.15.3:0/2815149048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 msgr2=0x5595a06e4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06e4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 -- 172.21.15.3:0/2815149048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a05f2130 con 0x5595a06d79d0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6137fe640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 0x7fa5fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa6127fc640 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa6080096c0 tx=0x7fa608031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ba33758e2ababa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa5fbfff640 1 -- 172.21.15.3:0/2296487137 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa60c0cf3f0 con 0x7fa6141082e0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.155+0000 7fa5f97fa640 1 -- 172.21.15.3:0/2815149048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608043020 con 0x5595a06d79d0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa5f97fa640 1 -- 172.21.15.3:0/2815149048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6080046c0 con 0x5595a06d79d0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa5f97fa640 1 -- 172.21.15.3:0/2815149048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608035070 con 0x5595a06d79d0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 -- 172.21.15.3:0/2815149048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 msgr2=0x5595a06d7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa6137fe640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 0x7fa5fc0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0a41c0 tx=0x7fa60c06b930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa6080096c0 tx=0x7fa608031400 comp rx=0 tx=0).stop 2024-09-12T02:18:51.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 -- 172.21.15.3:0/2815149048 shutdown_connections 2024-09-12T02:18:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06e4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 --2- 172.21.15.3:0/2815149048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06d7db0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 -- 172.21.15.3:0/2815149048 >> 172.21.15.3:0/2815149048 conn(0x5595a06d3f60 msgr2=0x5595a06d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 -- 172.21.15.3:0/2815149048 shutdown_connections 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.156+0000 7fa61b994740 1 -- 172.21.15.3:0/2815149048 wait complete. 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 -- start start 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 Processor -- start 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595a06e5780 con 0x5595a06d79d0 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595a05f8f20 con 0x5595a06e2bb0 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595a06e5900 con 0x5595a06d82f0 2024-09-12T02:18:51.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa61b994740 10 client.0 _ll_register_callbacks cb 0x5595a05c2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45146/0 (socket says 172.21.15.3:45146) 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 -- 172.21.15.3:0/161897109 learned_addr learned my addr 172.21.15.3:0/161897109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa612ffd640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 -- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 msgr2=0x5595a06df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 -- 172.21.15.3:0/161897109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 msgr2=0x5595a06dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.157+0000 7fa6127fc640 1 -- 172.21.15.3:0/161897109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a03af2b0 con 0x5595a06d79d0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa6127fc640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa608000c00 tx=0x7fa608004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8dcef611e6f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:51.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60804d020 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa608004d80 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6080046c0 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa608043070 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595a06f10d0 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6080046c0 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa60803d7b0 con 0x5595a06d79d0 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 10 client.15471 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.158+0000 7fa61b994740 20 client.15471 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a06f3240 0x5595a06f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa61b994740 4 client.15471 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:51.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595a064b600 con 0x5595a06f3240 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa612ffd640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a06f3240 0x5595a06f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa612ffd640 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a06f3240 0x5595a06f46b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710752be6b7722c0 server_cookie=41a499aeb43a01df in_seq=0 out_seq=0 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.159+0000 7fa5f6ff5640 10 client.15471 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/161897109 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595a064b600 con 0x5595a06f3240 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa5f6ff5640 10 client.15471 handle_command_reply: tid=0 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 client.15471 shutdown 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 msgr2=0x5595a06e2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa608000c00 tx=0x7fa608004570 comp rx=0 tx=0).stop 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 shutdown_connections 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a06f3240 0x5595a06f46b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa5f6ff5640 0 client.15471 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 --2- 172.21.15.3:0/161897109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06e2470 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 >> 172.21.15.3:0/161897109 conn(0x5595a06d3f60 msgr2=0x5595a06d9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 shutdown_connections 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 1 -- 172.21.15.3:0/161897109 wait complete. 2024-09-12T02:18:51.185 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.160+0000 7fa61b994740 20 client.15471 trim_cache size 0 max 16384 2024-09-12T02:18:51.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 Processor -- start 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 -- start start 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f29f0 con 0x5595a06d79d0 2024-09-12T02:18:51.190 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f30c0 con 0x5595a06e2bb0 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa61b994740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f3790 con 0x5595a06d82f0 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45154/0 (socket says 172.21.15.3:45154) 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.178+0000 7fa6127fc640 1 -- 172.21.15.3:0/1497180011 learned_addr learned my addr 172.21.15.3:0/1497180011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa612ffd640 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 -- 172.21.15.3:0/1497180011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 msgr2=0x5595a06de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 -- 172.21.15.3:0/1497180011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 msgr2=0x5595a06fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 -- 172.21.15.3:0/1497180011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a06e02e0 con 0x5595a06d79d0 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa608005420 tx=0x7fa608001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adedb376e9973d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa5f8ff9640 1 -- 172.21.15.3:0/1497180011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60804d070 con 0x5595a06d79d0 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa5f8ff9640 1 -- 172.21.15.3:0/1497180011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa608009780 con 0x5595a06d79d0 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.179+0000 7fa5f8ff9640 1 -- 172.21.15.3:0/1497180011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608035040 con 0x5595a06d79d0 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- 172.21.15.3:0/1497180011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 msgr2=0x5595a06f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa608005420 tx=0x7fa608001e70 comp rx=0 tx=0).stop 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- 172.21.15.3:0/1497180011 shutdown_connections 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a06de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- 172.21.15.3:0/1497180011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a06f46b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- 172.21.15.3:0/1497180011 >> 172.21.15.3:0/1497180011 conn(0x5595a06d3f60 msgr2=0x5595a06cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- 172.21.15.3:0/1497180011 shutdown_connections 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- 172.21.15.3:0/1497180011 wait complete. 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 -- start start 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 Processor -- start 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a075a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.180+0000 7fa61b994740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f4530 con 0x5595a06d79d0 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f3e60 con 0x5595a06e2bb0 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595a06f4c00 con 0x5595a06d82f0 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 10 client.0 _ll_register_callbacks cb 0x7fa60400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:51.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45170/0 (socket says 172.21.15.3:45170) 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 -- 172.21.15.3:0/1755764983 learned_addr learned my addr 172.21.15.3:0/1755764983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa612ffd640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a075a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 -- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 msgr2=0x5595a075a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a075a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 -- 172.21.15.3:0/1755764983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 msgr2=0x5595a06f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 -- 172.21.15.3:0/1755764983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a06f6ce0 con 0x5595a06d79d0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa6127fc640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5595a06f4e60 tx=0x7fa60803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7f6ccc4f8487c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa5f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:51.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608035020 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa60803a6a0 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa608009780 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa60804d070 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.181+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595a0761930 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa608009780 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa6080317b0 con 0x5595a06d79d0 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 10 client.15477 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 20 client.15477 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a0765e40 0x5595a0764b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:51.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 4 client.15477 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "status"}) -- 0x5595a0645b10 con 0x5595a0765e40 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.182+0000 7fa612ffd640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a0765e40 0x5595a0764b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa5f6ff5640 10 client.15477 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa612ffd640 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a0765e40 0x5595a0764b50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e6a7c45e8607125 server_cookie=ba8e01d397fecfd5 in_seq=0 out_seq=0 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa5f6ff5640 1 -- 172.21.15.3:0/1755764983 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5595a0645b10 con 0x5595a0765e40 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa5f6ff5640 10 client.15477 handle_command_reply: tid=0 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 client.15477 shutdown 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 msgr2=0x5595a075a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5595a06f4e60 tx=0x7fa60803f040 comp rx=0 tx=0).stop 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 shutdown_connections 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5595a0765e40 0x5595a0764b50 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5595a06e2bb0 0x5595a06f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa5f6ff5640 0 client.15477 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595a06d82f0 0x5595a075a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 --2- 172.21.15.3:0/1755764983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5595a06d79d0 0x5595a075a0e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.183+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 >> 172.21.15.3:0/1755764983 conn(0x5595a06d3f60 msgr2=0x5595a06ddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 shutdown_connections 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa61b994740 1 -- 172.21.15.3:0/1755764983 wait complete. 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa61b994740 20 client.15477 trim_cache size 0 max 16384 2024-09-12T02:18:51.197 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "c595fa10-70ab-11ef-bcea-c7b262605968", 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24508, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 141.067258098, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 29, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 83, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 160.65527649500001 2024-09-12T02:18:51.198 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:18:51.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 msgr2=0x7fa5fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 0x7fa5fc0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0a41c0 tx=0x7fa60c06b930 comp rx=0 tx=0).stop 2024-09-12T02:18:51.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 msgr2=0x7fa61419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:51.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00eaa0 tx=0x7fa60c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:18:51.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 shutdown_connections 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa5fc0a2c10 0x7fa5fc0a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6141082e0 0x7fa61419ce60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa614103760 0x7fa614198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 --2- 172.21.15.3:0/2296487137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614102db0 0x7fa614198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 >> 172.21.15.3:0/2296487137 conn(0x7fa6140fc7d0 msgr2=0x7fa614106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 shutdown_connections 2024-09-12T02:18:51.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:18:51.184+0000 7fa6195bb640 1 -- 172.21.15.3:0/2296487137 wait complete. 2024-09-12T02:18:51.873 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T02:18:52.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:52 smithi026 ceph-mon[35496]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-12T02:18:52.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:52 smithi003 ceph-mon[31476]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-12T02:18:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:52 smithi092 ceph-mon[36089]: pgmap v350: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-12T02:18:54.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:54 smithi026 ceph-mon[35496]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:54.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:54 smithi003 ceph-mon[31476]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:54.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:54 smithi092 ceph-mon[36089]: pgmap v351: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 -- 172.21.15.3:0/1172047481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc100a40 msgr2=0x7fadbc100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/1172047481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc100a40 0x7fadbc100e40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fadac009900 tx=0x7fadac02a410 comp rx=0 tx=0).stop 2024-09-12T02:18:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 -- 172.21.15.3:0/1172047481 shutdown_connections 2024-09-12T02:18:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/1172047481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 0x7fadbc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/1172047481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc101c30 0x7fadbc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/1172047481 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc100a40 0x7fadbc100e40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 -- 172.21.15.3:0/1172047481 >> 172.21.15.3:0/1172047481 conn(0x7fadbc0fc1c0 msgr2=0x7fadbc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 -- 172.21.15.3:0/1172047481 shutdown_connections 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.422+0000 7fadc0dd6640 1 -- 172.21.15.3:0/1172047481 wait complete. 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.423+0000 7fadc0dd6640 1 Processor -- start 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 -- start start 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc100a40 0x7fadbc19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 0x7fadbc1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10c6a0 con 0x7fadbc101c30 2024-09-12T02:18:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10c520 con 0x7fadbc100a40 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadc0dd6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc10c820 con 0x7fadbc1025f0 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45182/0 (socket says 172.21.15.3:45182) 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 -- 172.21.15.3:0/4289966762 learned_addr learned my addr 172.21.15.3:0/4289966762 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadba59c640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc100a40 0x7fadbc19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadbad9d640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 0x7fadbc1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 -- 172.21.15.3:0/4289966762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 msgr2=0x7fadbc1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 0x7fadbc1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 -- 172.21.15.3:0/4289966762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc100a40 msgr2=0x7fadbc19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc100a40 0x7fadbc19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.424+0000 7fadb1d9b640 1 -- 172.21.15.3:0/4289966762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1a1ca0 con 0x7fadbc101c30 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fada400c920 tx=0x7fada400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadb3fff640 1 -- 172.21.15.3:0/4289966762 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4007c70 con 0x7fadbc101c30 2024-09-12T02:18:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadb3fff640 1 -- 172.21.15.3:0/4289966762 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada4007e10 con 0x7fadbc101c30 2024-09-12T02:18:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1a1f90 con 0x7fadbc101c30 2024-09-12T02:18:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadb3fff640 1 -- 172.21.15.3:0/4289966762 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4018890 con 0x7fadbc101c30 2024-09-12T02:18:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.425+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a2450 con 0x7fadbc101c30 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.427+0000 7fadb3fff640 1 -- 172.21.15.3:0/4289966762 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fada4018a70 con 0x7fadbc101c30 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.427+0000 7fadc3c4e740 1 Processor -- start 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 -- start start 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66209da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a6620a370 0x562a6620c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a661682f0 con 0x562a662049a0 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a66110270 con 0x562a6620a370 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a661642b0 con 0x562a662052e0 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45198/0 (socket says 172.21.15.3:45198) 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 -- 172.21.15.3:0/941209006 learned_addr learned my addr 172.21.15.3:0/941209006 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadba59c640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66209da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a6620a370 0x562a6620c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 -- 172.21.15.3:0/941209006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 msgr2=0x562a66209da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66209da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 -- 172.21.15.3:0/941209006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a6620a370 msgr2=0x562a6620c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a6620a370 0x562a6620c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadbad9d640 1 -- 172.21.15.3:0/941209006 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6610f340 con 0x562a662049a0 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadba59c640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66209da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.428+0000 7fadb3fff640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadbad9d640 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fadb40039f0 tx=0x7fadb4003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46cbd058c2bde79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadb3fff640 1 -- 172.21.15.3:0/4289966762 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fada40cf270 con 0x7fadbc101c30 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadba59c640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadb0d99640 1 -- 172.21.15.3:0/941209006 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb401a470 con 0x562a662049a0 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadb0d99640 1 -- 172.21.15.3:0/941209006 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4015070 con 0x562a662049a0 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.429+0000 7fadb0d99640 1 -- 172.21.15.3:0/941209006 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4010900 con 0x562a662049a0 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- 172.21.15.3:0/941209006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 msgr2=0x562a66204da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadba59c640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 0x7fada80a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadac005ab0 tx=0x7fadac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fadb40039f0 tx=0x7fadb4003eb0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- 172.21.15.3:0/941209006 shutdown_connections 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a6620a370 0x562a6620c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66209da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/941209006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662049a0 0x562a66204da0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- 172.21.15.3:0/941209006 >> 172.21.15.3:0/941209006 conn(0x562a66200010 msgr2=0x562a66200410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- 172.21.15.3:0/941209006 shutdown_connections 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- 172.21.15.3:0/941209006 wait complete. 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 -- start start 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 Processor -- start 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.430+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a6620ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a6620f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562a6620d340 con 0x562a6620a370 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562a660f1290 con 0x562a662049a0 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562a6620d4c0 con 0x562a662052e0 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 10 client.0 _ll_register_callbacks cb 0x562a66122140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45202/0 (socket says 172.21.15.3:45202) 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 -- 172.21.15.3:0/1021969527 learned_addr learned my addr 172.21.15.3:0/1021969527 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadba59c640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a6620ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a6620f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 -- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 msgr2=0x562a6620f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a6620f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 -- 172.21.15.3:0/1021969527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 msgr2=0x562a6620ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a6620ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 -- 172.21.15.3:0/1021969527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a65e7b210 con 0x562a6620a370 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a6620f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadba59c640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a6620ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadbad9d640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fadb400a010 tx=0x7fadb4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b0cabbcad9211e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fada1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb402a060 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fadb4011710 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.431+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb4015070 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb4011910 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 10 client.15489 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a65f47620 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb4015070 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fadb401a570 con 0x562a6620a370 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 20 client.15489 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a6621c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 4 client.15489 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a66175a20 con 0x562a6621a5e0 2024-09-12T02:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.432+0000 7fadba59c640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a6621c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.433+0000 7fadba59c640 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a6621c9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3935aa2f0397b server_cookie=e9fae4658569b7fb in_seq=0 out_seq=0 2024-09-12T02:18:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.433+0000 7fada1ffb640 10 client.15489 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fada1ffb640 1 -- 172.21.15.3:0/1021969527 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a66175a20 con 0x562a6621a5e0 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fada1ffb640 10 client.15489 handle_command_reply: tid=0 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 client.15489 shutdown 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 msgr2=0x562a662161d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fadb400a010 tx=0x7fadb4005ae0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 shutdown_connections 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a6621c9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a662161d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a6620f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fada1ffb640 0 client.15489 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1021969527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a6620ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 >> 172.21.15.3:0/1021969527 conn(0x562a66200010 msgr2=0x562a66201820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 shutdown_connections 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1021969527 wait complete. 2024-09-12T02:18:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.434+0000 7fadc3c4e740 20 client.15489 trim_cache size 0 max 16384 2024-09-12T02:18:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 Processor -- start 2024-09-12T02:18:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 -- start start 2024-09-12T02:18:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a661ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 0x562a66201400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ab2e0 con 0x562a662052e0 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ace20 con 0x562a662049a0 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ab9b0 con 0x562a6620a370 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45218/0 (socket says 172.21.15.3:45218) 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 -- 172.21.15.3:0/1931600388 learned_addr learned my addr 172.21.15.3:0/1931600388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadbad9d640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 0x562a66201400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadba59c640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a661ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 -- 172.21.15.3:0/1931600388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 msgr2=0x562a66201400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 0x562a66201400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 -- 172.21.15.3:0/1931600388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 msgr2=0x562a661ff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a661ff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadb1d9b640 1 -- 172.21.15.3:0/1931600388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6620c760 con 0x562a662052e0 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadbad9d640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 0x562a66201400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.449+0000 7fadba59c640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a661ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fada4097dc0 tx=0x7fada400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd82941d19b2295 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fada3fff640 1 -- 172.21.15.3:0/1931600388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada409b960 con 0x562a662052e0 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fada3fff640 1 -- 172.21.15.3:0/1931600388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada409fd10 con 0x562a662052e0 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fada3fff640 1 -- 172.21.15.3:0/1931600388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada400fc50 con 0x562a662052e0 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1931600388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 msgr2=0x562a66216190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fada4097dc0 tx=0x7fada400ff30 comp rx=0 tx=0).stop 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1931600388 shutdown_connections 2024-09-12T02:18:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a6620a370 0x562a66201400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a662052e0 0x562a66216190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/1931600388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a661ff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1931600388 >> 172.21.15.3:0/1931600388 conn(0x562a66200010 msgr2=0x562a661fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1931600388 shutdown_connections 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.450+0000 7fadc3c4e740 1 -- 172.21.15.3:0/1931600388 wait complete. 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 -- start start 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 Processor -- start 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a66213cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66214200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ac750 con 0x562a6620a370 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1ac080 con 0x562a662049a0 2024-09-12T02:18:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1a92c0 con 0x562a662052e0 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadc3c4e740 10 client.0 _ll_register_callbacks cb 0x7fadb4009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45230/0 (socket says 172.21.15.3:45230) 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 -- 172.21.15.3:0/2657626855 learned_addr learned my addr 172.21.15.3:0/2657626855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadba59c640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a66213cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadb1d9b640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66214200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 -- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 msgr2=0x562a66214200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66214200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 -- 172.21.15.3:0/2657626855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 msgr2=0x562a66213cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a66213cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadbad9d640 1 -- 172.21.15.3:0/2657626855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a66215820 con 0x562a6620a370 2024-09-12T02:18:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.451+0000 7fadba59c640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a66213cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadbad9d640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fadb4002230 tx=0x7fadb4005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5796c2ad85c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb402a070 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada409fd10 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fadb4022040 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb403f050 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a662854d0 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada409fd10 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fadb4037020 con 0x562a6620a370 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:18:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 20 client.15495 populate_metadata read hostname 'smithi003' 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a66287f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 4 client.15495 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.452+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a6616fd70 con 0x562a6621a5e0 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.453+0000 7fadba59c640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a66287f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.453+0000 7fadba59c640 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a66287f10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf598d1423514e server_cookie=806c6ea89bfa857b in_seq=0 out_seq=0 2024-09-12T02:18:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.453+0000 7fada1ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.460+0000 7fada1ffb640 1 -- 172.21.15.3:0/2657626855 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a6616fd70 con 0x562a6621a5e0 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.460+0000 7fada1ffb640 10 client.15495 handle_command_reply: tid=0 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.460+0000 7fadc3c4e740 1 client.15495 shutdown 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 msgr2=0x562a66215210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fadb4002230 tx=0x7fadb4005df0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 shutdown_connections 2024-09-12T02:18:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562a6621a5e0 0x562a66287f10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562a6620a370 0x562a66215210 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562a662052e0 0x562a66214200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fada1ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 --2- 172.21.15.3:0/2657626855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562a662049a0 0x562a66213cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 >> 172.21.15.3:0/2657626855 conn(0x562a66200010 msgr2=0x562a661ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 shutdown_connections 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 1 -- 172.21.15.3:0/2657626855 wait complete. 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc3c4e740 20 client.15495 trim_cache size 0 max 16384 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:18:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:18:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:18:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 msgr2=0x7fada80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 0x7fada80a50d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadac005ab0 tx=0x7fadac000bc0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 msgr2=0x7fadbc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fada400c920 tx=0x7fada400cde0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 shutdown_connections 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fada80a2c10 0x7fada80a50d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fadbc1025f0 0x7fadbc1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fadbc101c30 0x7fadbc19a5c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 --2- 172.21.15.3:0/4289966762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadbc100a40 0x7fadbc19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.461+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 >> 172.21.15.3:0/4289966762 conn(0x7fadbc0fc1c0 msgr2=0x7fadbc069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.462+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 shutdown_connections 2024-09-12T02:18:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:18:55.462+0000 7fadc0dd6640 1 -- 172.21.15.3:0/4289966762 wait complete. 2024-09-12T02:18:56.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:56 smithi026 ceph-mon[35496]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:56.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:56 smithi003 ceph-mon[31476]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:56.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:56 smithi092 ceph-mon[36089]: pgmap v352: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-12T02:18:58.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:18:58 smithi026 ceph-mon[35496]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-09-12T02:18:58.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:18:58 smithi003 ceph-mon[31476]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-09-12T02:18:58.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:18:58 smithi092 ceph-mon[36089]: pgmap v353: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-09-12T02:19:00.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:00 smithi026 ceph-mon[35496]: pgmap v354: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:19:00.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:00 smithi003 ceph-mon[31476]: pgmap v354: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:19:00.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:00 smithi092 ceph-mon[36089]: pgmap v354: 97 pgs: 97 active+clean; 663 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:19:02.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:02 smithi026 ceph-mon[35496]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:02.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:02 smithi003 ceph-mon[31476]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:02.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:02 smithi092 ceph-mon[36089]: pgmap v355: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:02.975 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:19:03.239 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.744+0000 7fb739ff6640 1 -- 172.21.15.3:0/3223447921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734101de0 msgr2=0x7fb7341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.744+0000 7fb739ff6640 1 --2- 172.21.15.3:0/3223447921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734101de0 0x7fb7341021c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb72402a410 comp rx=0 tx=0).stop 2024-09-12T02:19:03.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 -- 172.21.15.3:0/3223447921 shutdown_connections 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 --2- 172.21.15.3:0/3223447921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734106f10 0x7fb73410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 --2- 172.21.15.3:0/3223447921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734102790 0x7fb7341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 --2- 172.21.15.3:0/3223447921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734101de0 0x7fb7341021c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 -- 172.21.15.3:0/3223447921 >> 172.21.15.3:0/3223447921 conn(0x7fb7340fd8e0 msgr2=0x7fb7340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 -- 172.21.15.3:0/3223447921 shutdown_connections 2024-09-12T02:19:03.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.745+0000 7fb739ff6640 1 -- 172.21.15.3:0/3223447921 wait complete. 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 Processor -- start 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 -- start start 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734101de0 0x7fb7341984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 0x7fb73419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb734111f70 con 0x7fb734102790 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb734111df0 con 0x7fb734101de0 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.746+0000 7fb739ff6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341120f0 con 0x7fb734106f10 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34650/0 (socket says 172.21.15.3:34650) 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 -- 172.21.15.3:0/1319535220 learned_addr learned my addr 172.21.15.3:0/1319535220 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb733fff640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 0x7fb73419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb7337fe640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734101de0 0x7fb7341984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 -- 172.21.15.3:0/1319535220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 msgr2=0x7fb73419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 0x7fb73419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 -- 172.21.15.3:0/1319535220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734101de0 msgr2=0x7fb7341984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734101de0 0x7fb7341984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 -- 172.21.15.3:0/1319535220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73419d460 con 0x7fb734102790 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb733fff640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 0x7fb73419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb732ffd640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e9f0 tx=0x7fb72800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.747+0000 7fb730ff9640 1 -- 172.21.15.3:0/1319535220 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72800cd80 con 0x7fb734102790 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.748+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73419d750 con 0x7fb734102790 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.748+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7341a5030 con 0x7fb734102790 2024-09-12T02:19:03.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.748+0000 7fb730ff9640 1 -- 172.21.15.3:0/1319535220 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728005500 con 0x7fb734102790 2024-09-12T02:19:03.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.748+0000 7fb730ff9640 1 -- 172.21.15.3:0/1319535220 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728010690 con 0x7fb734102790 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.750+0000 7fb73c3cf740 1 Processor -- start 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.750+0000 7fb73c3cf740 1 -- start start 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.750+0000 7fb730ff9640 1 -- 172.21.15.3:0/1319535220 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb728010830 con 0x7fb734102790 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.750+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbded2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdee3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562fbde382e0 con 0x562fbded2ca0 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562fbdddfce0 con 0x562fbded2380 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562fbde342a0 con 0x562fbdee1a90 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34660/0 (socket says 172.21.15.3:34660) 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 -- 172.21.15.3:0/3380611647 learned_addr learned my addr 172.21.15.3:0/3380611647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb733fff640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbded2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb732ffd640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdee3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 -- 172.21.15.3:0/3380611647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 msgr2=0x562fbdee3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdee3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 -- 172.21.15.3:0/3380611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 msgr2=0x562fbded2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbded2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb7337fe640 1 -- 172.21.15.3:0/3380611647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbddc4440 con 0x562fbded2ca0 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.751+0000 7fb733fff640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbded2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb730ff9640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 0x7fb71c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb724005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22380a049155218 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb730ff9640 1 -- 172.21.15.3:0/1319535220 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb7280cf3d0 con 0x7fb734102790 2024-09-12T02:19:03.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb733fff640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 0x7fb71c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb719ffb640 1 -- 172.21.15.3:0/3380611647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724040070 con 0x562fbded2ca0 2024-09-12T02:19:03.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb719ffb640 1 -- 172.21.15.3:0/3380611647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7240319f0 con 0x562fbded2ca0 2024-09-12T02:19:03.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.752+0000 7fb719ffb640 1 -- 172.21.15.3:0/3380611647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724035040 con 0x562fbded2ca0 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3380611647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 msgr2=0x562fbdee1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb733fff640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 0x7fb71c0a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb72c010880 tx=0x7fb72c00a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb724005e80 comp rx=0 tx=0).stop 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3380611647 shutdown_connections 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdee3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee1550 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3380611647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbded2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3380611647 >> 172.21.15.3:0/3380611647 conn(0x562fbdecea30 msgr2=0x562fbdecee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3380611647 shutdown_connections 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3380611647 wait complete. 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 -- start start 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.753+0000 7fb73c3cf740 1 Processor -- start 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 0x562fbdede600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdedeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562fbdee4660 con 0x562fbded2380 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562fbddf3050 con 0x562fbded2ca0 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562fbdee47e0 con 0x562fbdee1a90 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb73c3cf740 10 client.0 _ll_register_callbacks cb 0x562fbde33160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34668/0 (socket says 172.21.15.3:34668) 2024-09-12T02:19:03.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 -- 172.21.15.3:0/363445341 learned_addr learned my addr 172.21.15.3:0/363445341 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb732ffd640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 0x562fbdede600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 -- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 msgr2=0x562fbdedeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb733fff640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdedeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdedeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 -- 172.21.15.3:0/363445341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 msgr2=0x562fbdede600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 0x562fbdede600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb7337fe640 1 -- 172.21.15.3:0/363445341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbdba93c0 con 0x562fbded2380 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb732ffd640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 0x562fbdede600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.754+0000 7fb733fff640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdedeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb7337fe640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a450 tx=0x7fb724005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac571d8c65550d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72404d060 con 0x562fbded2380 2024-09-12T02:19:03.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb724004240 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7240319f0 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb724040070 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb73c3cf740 10 client.15507 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fbdef0050 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7240319f0 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.755+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7240038c0 con 0x562fbded2380 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 10 client.15507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 20 client.15507 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdef13a0 0x562fbdef3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 4 client.15507 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:03.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fbde45850 con 0x562fbdef13a0 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb732ffd640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdef13a0 0x562fbdef3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb712ffd640 10 client.15507 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.756+0000 7fb732ffd640 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdef13a0 0x562fbdef3790 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a63d021245b486e9 server_cookie=3442fb3de1ee12d4 in_seq=0 out_seq=0 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb712ffd640 1 -- 172.21.15.3:0/363445341 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fbde45850 con 0x562fbdef13a0 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb712ffd640 10 client.15507 handle_command_reply: tid=0 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 client.15507 shutdown 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 msgr2=0x562fbdee1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a450 tx=0x7fb724005250 comp rx=0 tx=0).stop 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 shutdown_connections 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdef13a0 0x562fbdef3790 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbdedeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2ca0 0x562fbdede600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/363445341 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2380 0x562fbdee1350 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 >> 172.21.15.3:0/363445341 conn(0x562fbdecea30 msgr2=0x562fbded4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.757+0000 7fb712ffd640 0 client.15507 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:03.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.758+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 shutdown_connections 2024-09-12T02:19:03.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.758+0000 7fb73c3cf740 1 -- 172.21.15.3:0/363445341 wait complete. 2024-09-12T02:19:03.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.758+0000 7fb73c3cf740 20 client.15507 trim_cache size 0 max 16384 2024-09-12T02:19:03.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.776+0000 7fb73c3cf740 1 Processor -- start 2024-09-12T02:19:03.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.776+0000 7fb73c3cf740 1 -- start start 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbdeed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbded9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000e480 con 0x562fbded2ca0 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000ffc0 con 0x562fbded2380 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000eb50 con 0x562fbdee1a90 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34672/0 (socket says 172.21.15.3:34672) 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 -- 172.21.15.3:0/972717307 learned_addr learned my addr 172.21.15.3:0/972717307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb733fff640 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbded9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb7337fe640 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbdeed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 -- 172.21.15.3:0/972717307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 msgr2=0x562fbded9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbded9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 -- 172.21.15.3:0/972717307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 msgr2=0x562fbdeed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbdeed960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 -- 172.21.15.3:0/972717307 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fbdedd5b0 con 0x562fbded2ca0 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.777+0000 7fb732ffd640 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c6a0 tx=0x7fb7280d18c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a9c9fb059a296 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.778+0000 7fb7197fa640 1 -- 172.21.15.3:0/972717307 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72809b420 con 0x562fbded2ca0 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.778+0000 7fb7197fa640 1 -- 172.21.15.3:0/972717307 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72809c1c0 con 0x562fbded2ca0 2024-09-12T02:19:03.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.778+0000 7fb7197fa640 1 -- 172.21.15.3:0/972717307 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728004a40 con 0x562fbded2ca0 2024-09-12T02:19:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- 172.21.15.3:0/972717307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 msgr2=0x562fbdede7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb72800c6a0 tx=0x7fb7280d18c0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- 172.21.15.3:0/972717307 shutdown_connections 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbdee1a90 0x562fbded9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdede7f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/972717307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbded2380 0x562fbdeed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- 172.21.15.3:0/972717307 >> 172.21.15.3:0/972717307 conn(0x562fbdecea30 msgr2=0x562fbded94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- 172.21.15.3:0/972717307 shutdown_connections 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- 172.21.15.3:0/972717307 wait complete. 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 -- start start 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.779+0000 7fb73c3cf740 1 Processor -- start 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbded2380 0x562fbdeea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbdee1a90 0x562fbdee5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000f8f0 con 0x562fbded2ca0 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000f220 con 0x562fbdee1a90 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb72000c460 con 0x562fbded2380 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb73c3cf740 10 client.0 _ll_register_callbacks cb 0x7fb72c00aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34680/0 (socket says 172.21.15.3:34680) 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 -- 172.21.15.3:0/3922035752 learned_addr learned my addr 172.21.15.3:0/3922035752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbded2380 0x562fbdeea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb733fff640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbdee1a90 0x562fbdee5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 -- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbded2380 msgr2=0x562fbdeea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbded2380 0x562fbdeea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 -- 172.21.15.3:0/3922035752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbdee1a90 msgr2=0x562fbdee5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:03.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbdee1a90 0x562fbdee5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 -- 172.21.15.3:0/3922035752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fbdee5e80 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb732ffd640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb7280ccdf0 tx=0x7fb72809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bfa7d4a8498d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.780+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280b4060 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb7280ccfc0 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72809c1c0 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb72809fe70 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562fbdf5bc20 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72809c1c0 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7280a42a0 con 0x562fbded2ca0 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:03.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 20 client.15513 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdf5c330 0x562fbdf5e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 4 client.15513 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fbdedeed0 con 0x562fbdf5c330 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.781+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdf5c330 0x562fbdf5e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb7337fe640 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdf5c330 0x562fbdf5e750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707f91cccc7dc7e6 server_cookie=af1b87273420c714 in_seq=0 out_seq=0 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb712ffd640 10 client.15513 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb712ffd640 1 -- 172.21.15.3:0/3922035752 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562fbdedeed0 con 0x562fbdf5c330 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb712ffd640 10 client.15513 handle_command_reply: tid=0 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 client.15513 shutdown 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 msgr2=0x562fbdee5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb7280ccdf0 tx=0x7fb72809b530 comp rx=0 tx=0).stop 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 shutdown_connections 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562fbdf5c330 0x562fbdf5e750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562fbdee1a90 0x562fbdee5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562fbded2ca0 0x562fbdee5350 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 --2- 172.21.15.3:0/3922035752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562fbded2380 0x562fbdeea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 >> 172.21.15.3:0/3922035752 conn(0x562fbdecea30 msgr2=0x562fbdecd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb712ffd640 0 client.15513 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 shutdown_connections 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.782+0000 7fb73c3cf740 1 -- 172.21.15.3:0/3922035752 wait complete. 2024-09-12T02:19:03.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb73c3cf740 20 client.15513 trim_cache size 0 max 16384 2024-09-12T02:19:03.788 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:03.788 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-12T02:19:03.788 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-12T02:19:03.788 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:03.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 msgr2=0x7fb71c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 0x7fb71c0a5150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb72c010880 tx=0x7fb72c00a4d0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 msgr2=0x7fb7341989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e9f0 tx=0x7fb72800edf0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 shutdown_connections 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb71c0a2c90 0x7fb71c0a5150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb734106f10 0x7fb73419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb734102790 0x7fb7341989f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 --2- 172.21.15.3:0/1319535220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb734101de0 0x7fb7341984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 >> 172.21.15.3:0/1319535220 conn(0x7fb7340fd8e0 msgr2=0x7fb7340fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 shutdown_connections 2024-09-12T02:19:03.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:03.783+0000 7fb739ff6640 1 -- 172.21.15.3:0/1319535220 wait complete. 2024-09-12T02:19:04.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:04 smithi026 ceph-mon[35496]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:04.524 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:04 smithi003 ceph-mon[31476]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:04.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:04 smithi092 ceph-mon[36089]: pgmap v356: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:04.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:19:04.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:19:04.820 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:06.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:06 smithi026 ceph-mon[35496]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:06.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:06 smithi003 ceph-mon[31476]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:06.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:06 smithi092 ceph-mon[36089]: pgmap v357: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:19:06.875 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T02:19:06.875 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 -- 172.21.15.3:0/3579494484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 msgr2=0x7f28b8100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3579494484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8100d00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009900 tx=0x7f28a802a410 comp rx=0 tx=0).stop 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 -- 172.21.15.3:0/3579494484 shutdown_connections 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3579494484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3579494484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 0x7f28b810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3579494484 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8100d00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.987+0000 7f28be3e9640 1 -- 172.21.15.3:0/3579494484 >> 172.21.15.3:0/3579494484 conn(0x7f28b8075f70 msgr2=0x7f28b8076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.988+0000 7f28be3e9640 1 -- 172.21.15.3:0/3579494484 shutdown_connections 2024-09-12T02:19:07.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.988+0000 7f28be3e9640 1 -- 172.21.15.3:0/3579494484 wait complete. 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.988+0000 7f28be3e9640 1 Processor -- start 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 -- start start 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 0x7f28b8103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8114160 con 0x7f28b8100920 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8113fe0 con 0x7f28b810e400 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28be3e9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81142e0 con 0x7f28b8101240 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34690/0 (socket says 172.21.15.3:34690) 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.989+0000 7f28b7fff640 1 -- 172.21.15.3:0/3958687471 learned_addr learned my addr 172.21.15.3:0/3958687471 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:07.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b77fe640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 0x7f28b8103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 -- 172.21.15.3:0/3958687471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 msgr2=0x7f28b8103ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 0x7f28b8103ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 -- 172.21.15.3:0/3958687471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 msgr2=0x7f28b8104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28bc986640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 -- 172.21.15.3:0/3958687471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81aace0 con 0x7f28b8100920 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28bc986640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.990+0000 7f28b7fff640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.991+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a803f030 con 0x7f28b8100920 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.991+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a8003ce0 con 0x7f28b8100920 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.991+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b81aaf70 con 0x7f28b8100920 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.991+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81ab420 con 0x7f28b8100920 2024-09-12T02:19:07.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.992+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035070 con 0x7f28b8100920 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.992+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b806a180 con 0x7f28b8100920 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.994+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f28a8042030 con 0x7f28b8100920 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.995+0000 7f28b57fa640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 0x7f28a00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.996+0000 7f28b77fe640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 0x7f28a00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.996+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f28a80ef610 con 0x7f28b8100920 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:06.996+0000 7f28b77fe640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 0x7f28a00a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28b8105270 tx=0x7f28ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:07.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.001+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28a80bc460 con 0x7f28b8100920 2024-09-12T02:19:07.188 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.278+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b8100d00 con 0x7f28b8100920 2024-09-12T02:19:07.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.279+0000 7f28b57fa640 1 -- 172.21.15.3:0/3958687471 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f28a80bbd60 con 0x7f28b8100920 2024-09-12T02:19:07.280 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:19:07.280 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:19:07.280 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:19:07.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 msgr2=0x7f28a00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:07.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 0x7f28a00a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f28b8105270 tx=0x7f28ac00b410 comp rx=0 tx=0).stop 2024-09-12T02:19:07.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 msgr2=0x7f28b8103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:07.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f28a8000c00 tx=0x7f28a80042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.282 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 shutdown_connections 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f28a00a2c90 0x7f28a00a5150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b810e400 0x7f28b8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28b8101240 0x7f28b8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 --2- 172.21.15.3:0/3958687471 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f28b8100920 0x7f28b8103ab0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 >> 172.21.15.3:0/3958687471 conn(0x7f28b8075f70 msgr2=0x7f28b810e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 shutdown_connections 2024-09-12T02:19:07.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:07.282+0000 7f28be3e9640 1 -- 172.21.15.3:0/3958687471 wait complete. 2024-09-12T02:19:08.151 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:19:08.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 -- 172.21.15.3:0/3794044487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 msgr2=0x7f5ee01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/3794044487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee01089f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009900 tx=0x7f5ed002a410 comp rx=0 tx=0).stop 2024-09-12T02:19:08.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 -- 172.21.15.3:0/3794044487 shutdown_connections 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/3794044487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/3794044487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee01089f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/3794044487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 0x7f5ee01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 -- 172.21.15.3:0/3794044487 >> 172.21.15.3:0/3794044487 conn(0x7f5ee00ffab0 msgr2=0x7f5ee0101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 -- 172.21.15.3:0/3794044487 shutdown_connections 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.321+0000 7f5ee84ad640 1 -- 172.21.15.3:0/3794044487 wait complete. 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 Processor -- start 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 -- start start 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 0x7f5ee019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0114120 con 0x7f5ee01049a0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee0113fa0 con 0x7f5ee0109120 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee84ad640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee01142a0 con 0x7f5ee0103ff0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee5a48640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee5a48640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34710/0 (socket says 172.21.15.3:34710) 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee5a48640 1 -- 172.21.15.3:0/1621583357 learned_addr learned my addr 172.21.15.3:0/1621583357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee6249640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 0x7f5ee019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.322+0000 7f5ee6a4a640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 -- 172.21.15.3:0/1621583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 msgr2=0x7f5ee019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 0x7f5ee019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 -- 172.21.15.3:0/1621583357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 msgr2=0x7f5ee019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 -- 172.21.15.3:0/1621583357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee019f5f0 con 0x7f5ee01049a0 2024-09-12T02:19:08.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee6a4a640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee5a48640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006010 tx=0x7f5ed00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed003f030 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ed0003ce0 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee019f880 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0035070 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.323+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee01a71c0 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.324+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee01043d0 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.327+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5ed0042030 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.328+0000 7f5ed77fe640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 0x7f5ecc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.328+0000 7f5ee6249640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 0x7f5ecc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.328+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5ed00efa40 con 0x7f5ee01049a0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.328+0000 7f5ee6249640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 0x7f5ecc0a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00bbe0 tx=0x7f5edc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.331+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ed00bc7d0 con 0x7f5ee01049a0 2024-09-12T02:19:08.445 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:08 smithi026 ceph-mon[35496]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:08 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3958687471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:08.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:08 smithi003 ceph-mon[31476]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:08 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3958687471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:08 smithi092 ceph-mon[36089]: pgmap v358: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-12T02:19:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:08 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3958687471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 -- 172.21.15.3:0/2243713869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa310107820 msgr2=0x7fa310109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 --2- 172.21.15.3:0/2243713869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa310107820 0x7fa310109c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa308009ca0 tx=0x7fa30802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 -- 172.21.15.3:0/2243713869 shutdown_connections 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 --2- 172.21.15.3:0/2243713869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa310107820 0x7fa310109c90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 --2- 172.21.15.3:0/2243713869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 --2- 172.21.15.3:0/2243713869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3101025c0 0x7fa3101049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 -- 172.21.15.3:0/2243713869 >> 172.21.15.3:0/2243713869 conn(0x7fa3100fc1c0 msgr2=0x7fa3100fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 -- 172.21.15.3:0/2243713869 shutdown_connections 2024-09-12T02:19:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.559+0000 7fa317552640 1 -- 172.21.15.3:0/2243713869 wait complete. 2024-09-12T02:19:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 Processor -- start 2024-09-12T02:19:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 -- start start 2024-09-12T02:19:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 0x7fa31019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa31010d7e0 con 0x7fa3101025c0 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa31010d660 con 0x7fa310104ef0 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa317552640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa31010d960 con 0x7fa310107820 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44946/0 (socket says 172.21.15.3:44946) 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.560+0000 7fa3152ee640 1 -- 172.21.15.3:0/1261756232 learned_addr learned my addr 172.21.15.3:0/1261756232 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa314aed640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa315aef640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 0x7fa31019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 -- 172.21.15.3:0/1261756232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 msgr2=0x7fa31019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 0x7fa31019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 -- 172.21.15.3:0/1261756232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 msgr2=0x7fa3101983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 -- 172.21.15.3:0/1261756232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31019fac0 con 0x7fa3101025c0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa314aed640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa315aef640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 0x7fa31019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00d950 tx=0x7fa30c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3067fc640 1 -- 172.21.15.3:0/1261756232 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c017030 con 0x7fa3101025c0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3067fc640 1 -- 172.21.15.3:0/1261756232 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c004020 con 0x7fa3101025c0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31019fdb0 con 0x7fa3101025c0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa3067fc640 1 -- 172.21.15.3:0/1261756232 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0047a0 con 0x7fa3101025c0 2024-09-12T02:19:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.561+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3101a0220 con 0x7fa3101025c0 2024-09-12T02:19:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.564+0000 7fa3067fc640 1 -- 172.21.15.3:0/1261756232 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa30c0041d0 con 0x7fa3101025c0 2024-09-12T02:19:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.564+0000 7fa31a3ca740 1 Processor -- start 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 -- start start 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc9944da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc9945370 0x55edc9947830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55edc98a32f0 con 0x55edc993f9a0 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55edc984b270 con 0x55edc9945370 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55edc989f2b0 con 0x55edc99402e0 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44954/0 (socket says 172.21.15.3:44954) 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 -- 172.21.15.3:0/1249630040 learned_addr learned my addr 172.21.15.3:0/1249630040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc9944da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa314aed640 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc9945370 0x55edc9947830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa3067fc640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 0x7fa2fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 -- 172.21.15.3:0/1249630040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 msgr2=0x55edc9944da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc9944da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 -- 172.21.15.3:0/1249630040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc9945370 msgr2=0x55edc9947830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc9945370 0x55edc9947830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.565+0000 7fa315aef640 1 -- 172.21.15.3:0/1249630040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc984a340 con 0x55edc993f9a0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa3067fc640 1 -- 172.21.15.3:0/1261756232 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa30c0ced30 con 0x7fa3101025c0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa315aef640 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa308009ca0 tx=0x7fa3080093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ecb94aa7405ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa315aef640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 0x7fa2fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa2fb7fe640 1 -- 172.21.15.3:0/1249630040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308031450 con 0x55edc993f9a0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa2fb7fe640 1 -- 172.21.15.3:0/1249630040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa308036050 con 0x55edc993f9a0 2024-09-12T02:19:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.566+0000 7fa2fb7fe640 1 -- 172.21.15.3:0/1249630040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30802f610 con 0x55edc993f9a0 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1249630040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 msgr2=0x55edc993fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa315aef640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 0x7fa2fc0a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa308049000 tx=0x7fa30804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa308009ca0 tx=0x7fa3080093e0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1249630040 shutdown_connections 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc9945370 0x55edc9947830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc9944da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1249630040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fda0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1249630040 >> 172.21.15.3:0/1249630040 conn(0x55edc993b010 msgr2=0x55edc993b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1249630040 shutdown_connections 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1249630040 wait complete. 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.567+0000 7fa31a3ca740 1 -- start start 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 Processor -- start 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9949fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc9950f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55edc99481c0 con 0x55edc993f9a0 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55edc982c290 con 0x55edc99402e0 2024-09-12T02:19:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55edc9948340 con 0x55edc9945370 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa31a3ca740 10 client.0 _ll_register_callbacks cb 0x55edc985d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44956/0 (socket says 172.21.15.3:44956) 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa3152ee640 1 -- 172.21.15.3:0/1427459081 learned_addr learned my addr 172.21.15.3:0/1427459081 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa315aef640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc9950f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.568+0000 7fa314aed640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9949fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 -- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 msgr2=0x55edc9950f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc9950f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 -- 172.21.15.3:0/1427459081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 msgr2=0x55edc9949fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9949fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 -- 172.21.15.3:0/1427459081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc95b6210 con 0x55edc993f9a0 2024-09-12T02:19:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa315aef640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc9950f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009610 tx=0x7fa30c0d23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5486a1fb53cfe280 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa2f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c09b1f0 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa30c0a3c00 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa308036050 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa30c09ad80 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa31a3ca740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55edc9682620 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.569+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa308036050 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa30c0ccce0 con 0x55edc993f9a0 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 20 client.15537 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc9957af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 4 client.15537 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edc98b0a20 con 0x55edc99556e0 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.570+0000 7fa314aed640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc9957af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.571+0000 7fa2f8ff9640 10 client.15537 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.571+0000 7fa314aed640 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc9957af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf3151d0e15520c server_cookie=b13362a004552534 in_seq=0 out_seq=0 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.571+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1427459081 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edc98b0a20 con 0x55edc99556e0 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.571+0000 7fa2f8ff9640 10 client.15537 handle_command_reply: tid=0 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.571+0000 7fa31a3ca740 1 client.15537 shutdown 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 msgr2=0x55edc9949a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009610 tx=0x7fa30c0d23b0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 shutdown_connections 2024-09-12T02:19:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc9957af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc9950f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa2f8ff9640 0 client.15537 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9949fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1427459081 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc9949a80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 >> 172.21.15.3:0/1427459081 conn(0x55edc993b010 msgr2=0x55edc993c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 shutdown_connections 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1427459081 wait complete. 2024-09-12T02:19:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.572+0000 7fa31a3ca740 20 client.15537 trim_cache size 0 max 16384 2024-09-12T02:19:08.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.592+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ee019b9a0 con 0x7f5ee01049a0 2024-09-12T02:19:08.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.592+0000 7f5ed77fe640 1 -- 172.21.15.3:0/1621583357 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f5ee019b9a0 con 0x7f5ee01049a0 2024-09-12T02:19:08.594 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:19:08.594 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:19:08.594 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:19:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 msgr2=0x7f5ecc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 0x7f5ecc0a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00bbe0 tx=0x7f5edc00b390 comp rx=0 tx=0).stop 2024-09-12T02:19:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 msgr2=0x7f5ee019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0006010 tx=0x7f5ed00042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 shutdown_connections 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5ecc0a2c10 0x7f5ecc0a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0109120 0x7f5ee019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee01049a0 0x7f5ee019ab80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 --2- 172.21.15.3:0/1621583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ee0103ff0 0x7f5ee019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 >> 172.21.15.3:0/1621583357 conn(0x7f5ee00ffab0 msgr2=0x7f5ee0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 shutdown_connections 2024-09-12T02:19:08.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.596+0000 7f5ee84ad640 1 -- 172.21.15.3:0/1621583357 wait complete. 2024-09-12T02:19:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 Processor -- start 2024-09-12T02:19:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 -- start start 2024-09-12T02:19:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9950f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc993bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101a9090 con 0x55edc993f9a0 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101aabd0 con 0x55edc99402e0 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101a9760 con 0x55edc9945370 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa3152ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44972/0 (socket says 172.21.15.3:44972) 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.600+0000 7fa3152ee640 1 -- 172.21.15.3:0/3189337924 learned_addr learned my addr 172.21.15.3:0/3189337924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa314aed640 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9950f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 -- 172.21.15.3:0/3189337924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 msgr2=0x55edc993bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc993bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 -- 172.21.15.3:0/3189337924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 msgr2=0x55edc9950f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9950f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 -- 172.21.15.3:0/3189337924 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edc993ae30 con 0x55edc993f9a0 2024-09-12T02:19:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa3152ee640 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa30c097df0 tx=0x7fa30c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c793673e1e73ef20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa2faffd640 1 -- 172.21.15.3:0/3189337924 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c014070 con 0x55edc993f9a0 2024-09-12T02:19:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa2faffd640 1 -- 172.21.15.3:0/3189337924 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c09bb20 con 0x55edc993f9a0 2024-09-12T02:19:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.601+0000 7fa2faffd640 1 -- 172.21.15.3:0/3189337924 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c09af60 con 0x55edc993f9a0 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 -- 172.21.15.3:0/3189337924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 msgr2=0x55edc993fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa30c097df0 tx=0x7fa30c0057b0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 -- 172.21.15.3:0/3189337924 shutdown_connections 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc9945370 0x55edc993bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc99402e0 0x55edc9950f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/3189337924 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc993f9a0 0x55edc993fd80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 -- 172.21.15.3:0/3189337924 >> 172.21.15.3:0/3189337924 conn(0x55edc993b010 msgr2=0x55edc9949830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 -- 172.21.15.3:0/3189337924 shutdown_connections 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.602+0000 7fa31a3ca740 1 -- 172.21.15.3:0/3189337924 wait complete. 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 -- start start 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 Processor -- start 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 0x55edc994e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc994e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101aa500 con 0x55edc9945370 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101a9e30 con 0x55edc993f9a0 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101a7070 con 0x55edc99402e0 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa31a3ca740 10 client.0 _ll_register_callbacks cb 0x7fa30c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa315aef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa315aef640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44988/0 (socket says 172.21.15.3:44988) 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa315aef640 1 -- 172.21.15.3:0/1305704037 learned_addr learned my addr 172.21.15.3:0/1305704037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.603+0000 7fa314aed640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc994e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 0x55edc994e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 -- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 msgr2=0x55edc994e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc994e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 -- 172.21.15.3:0/1305704037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 msgr2=0x55edc994e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 0x55edc994e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 -- 172.21.15.3:0/1305704037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edc994fcb0 con 0x55edc9945370 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 0x55edc994e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa315aef640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa30802ade0 tx=0x7fa308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e7d3d40a6b1ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa2f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30802f610 con 0x55edc9945370 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa30802fb20 con 0x55edc9945370 2024-09-12T02:19:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.604+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c09bb20 con 0x55edc9945370 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa308004e50 con 0x55edc9945370 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55edc9951030 con 0x55edc9945370 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c09bb20 con 0x55edc9945370 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa308030aa0 con 0x55edc9945370 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 20 client.15543 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc99c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 4 client.15543 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edc98aad70 con 0x55edc99556e0 2024-09-12T02:19:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.605+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc99c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.606+0000 7fa3152ee640 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc99c6ab0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd057b76f1c6647a server_cookie=3f59dc054b91f36 in_seq=0 out_seq=0 2024-09-12T02:19:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.606+0000 7fa2f8ff9640 10 client.15543 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.609+0000 7fa2f8ff9640 1 -- 172.21.15.3:0/1305704037 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edc98aad70 con 0x55edc99556e0 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.609+0000 7fa2f8ff9640 10 client.15543 handle_command_reply: tid=0 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.609+0000 7fa31a3ca740 1 client.15543 shutdown 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 msgr2=0x55edc994f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa30802ade0 tx=0x7fa308004270 comp rx=0 tx=0).stop 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 shutdown_connections 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55edc99556e0 0x55edc99c6ab0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55edc9945370 0x55edc994f6a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55edc99402e0 0x55edc994e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 --2- 172.21.15.3:0/1305704037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55edc993f9a0 0x55edc994e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 >> 172.21.15.3:0/1305704037 conn(0x55edc993b010 msgr2=0x55edc99399a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa2f8ff9640 0 client.15543 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 shutdown_connections 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 1 -- 172.21.15.3:0/1305704037 wait complete. 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.610+0000 7fa31a3ca740 20 client.15543 trim_cache size 0 max 16384 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:19:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:19:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 msgr2=0x7fa2fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 0x7fa2fc0a50d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa308049000 tx=0x7fa30804a000 comp rx=0 tx=0).stop 2024-09-12T02:19:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 msgr2=0x7fa310197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00d950 tx=0x7fa30c00de10 comp rx=0 tx=0).stop 2024-09-12T02:19:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 shutdown_connections 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa2fc0a2c10 0x7fa2fc0a50d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa310107820 0x7fa31019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa310104ef0 0x7fa3101983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 --2- 172.21.15.3:0/1261756232 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa3101025c0 0x7fa310197ea0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 >> 172.21.15.3:0/1261756232 conn(0x7fa3100fc1c0 msgr2=0x7fa310102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 shutdown_connections 2024-09-12T02:19:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:08.611+0000 7fa317552640 1 -- 172.21.15.3:0/1261756232 wait complete. 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 -- 172.21.15.3:0/1399692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 msgr2=0x7f3c6c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1399692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c1021c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009900 tx=0x7f3c6802a410 comp rx=0 tx=0).stop 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 -- 172.21.15.3:0/1399692906 shutdown_connections 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1399692906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c106f10 0x7f3c6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1399692906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c102790 0x7f3c6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1399692906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c1021c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 -- 172.21.15.3:0/1399692906 >> 172.21.15.3:0/1399692906 conn(0x7f3c6c0fd8c0 msgr2=0x7f3c6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.969+0000 7f3c747c7640 1 -- 172.21.15.3:0/1399692906 shutdown_connections 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 -- 172.21.15.3:0/1399692906 wait complete. 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 Processor -- start 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 -- start start 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c102790 0x7f3c6c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c106f10 0x7f3c6c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c111f70 con 0x7f3c6c101de0 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c111df0 con 0x7f3c6c106f10 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.970+0000 7f3c747c7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1120f0 con 0x7f3c6c102790 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45002/0 (socket says 172.21.15.3:45002) 2024-09-12T02:19:08.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 -- 172.21.15.3:0/1982116900 learned_addr learned my addr 172.21.15.3:0/1982116900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c106f10 0x7f3c6c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c102790 0x7f3c6c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 -- 172.21.15.3:0/1982116900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c102790 msgr2=0x7f3c6c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c102790 0x7f3c6c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 -- 172.21.15.3:0/1982116900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c106f10 msgr2=0x7f3c6c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c106f10 0x7f3c6c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 -- 172.21.15.3:0/1982116900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6c19f5e0 con 0x7f3c6c101de0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c72563640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006010 tx=0x7f3c680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c637fe640 1 -- 172.21.15.3:0/1982116900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6803f030 con 0x7f3c6c101de0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6c19f870 con 0x7f3c6c101de0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c637fe640 1 -- 172.21.15.3:0/1982116900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c68003ce0 con 0x7f3c6c101de0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.971+0000 7f3c637fe640 1 -- 172.21.15.3:0/1982116900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68035070 con 0x7f3c6c101de0 2024-09-12T02:19:08.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.972+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c6c1a71b0 con 0x7f3c6c101de0 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.974+0000 7f3c637fe640 1 -- 172.21.15.3:0/1982116900 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3c68042030 con 0x7f3c6c101de0 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.974+0000 7f3c76ba0740 1 Processor -- start 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.974+0000 7f3c76ba0740 1 -- start start 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c6906b40 0x5555c6905090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6912760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555c686b2f0 con 0x5555c69055d0 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5555c6812cf0 con 0x5555c6910370 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555c68672b0 con 0x5555c6906b40 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45006/0 (socket says 172.21.15.3:45006) 2024-09-12T02:19:08.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 -- 172.21.15.3:0/1290597036 learned_addr learned my addr 172.21.15.3:0/1290597036 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6912760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c6906b40 0x5555c6905090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c637fe640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 0x7f3c580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 -- 172.21.15.3:0/1290597036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c6906b40 msgr2=0x5555c6905090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c6906b40 0x5555c6905090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 -- 172.21.15.3:0/1290597036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 msgr2=0x5555c6912760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 0x7f3c580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6912760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c72563640 1 -- 172.21.15.3:0/1290597036 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c67f73a0 con 0x5555c69055d0 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6912760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.975+0000 7f3c637fe640 1 -- 172.21.15.3:0/1982116900 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f3c680ef2f0 con 0x7f3c6c101de0 2024-09-12T02:19:08.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.976+0000 7f3c72563640 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3c680ecc00 tx=0x7f3c68031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe06633632d15d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.976+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 0x7f3c580a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009fd0 tx=0x7f3c64004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.976+0000 7f3c60ff9640 1 -- 172.21.15.3:0/1290597036 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68049030 con 0x5555c69055d0 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.976+0000 7f3c60ff9640 1 -- 172.21.15.3:0/1290597036 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6803e780 con 0x5555c69055d0 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.976+0000 7f3c60ff9640 1 -- 172.21.15.3:0/1290597036 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c680bc1a0 con 0x5555c69055d0 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1290597036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 msgr2=0x5555c6905a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3c680ecc00 tx=0x7f3c68031a60 comp rx=0 tx=0).stop 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1290597036 shutdown_connections 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6912760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c69055d0 0x5555c6905a10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1290597036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c6906b40 0x5555c6905090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1290597036 >> 172.21.15.3:0/1290597036 conn(0x5555c68ce310 msgr2=0x5555c6901c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1290597036 shutdown_connections 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1290597036 wait complete. 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 -- start start 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 Processor -- start 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c690fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6906b40 0x5555c690cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.977+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555c686b2f0 con 0x5555c6910370 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5555c6826060 con 0x5555c6906b40 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555c6912e30 con 0x5555c69055d0 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c76ba0740 10 client.0 _ll_register_callbacks cb 0x5555c65798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45014/0 (socket says 172.21.15.3:45014) 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 -- 172.21.15.3:0/1053438116 learned_addr learned my addr 172.21.15.3:0/1053438116 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6906b40 0x5555c690cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72563640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c690fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 -- 172.21.15.3:0/1053438116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 msgr2=0x5555c690fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c690fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 -- 172.21.15.3:0/1053438116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6906b40 msgr2=0x5555c690cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6906b40 0x5555c690cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 -- 172.21.15.3:0/1053438116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c65dc3c0 con 0x5555c6910370 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72563640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c690fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3c64012040 tx=0x7f3c64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de079e9c146162fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.978+0000 7f3c55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64025070 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f3c64010dc0 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6803e780 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c6402a460 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555c691e770 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6803e780 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6401f570 con 0x5555c6910370 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 10 client.15555 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 20 client.15555 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:08.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c691fcd0 0x5555c6922100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 4 client.15555 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:08.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.979+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c6878860 con 0x5555c691fcd0 2024-09-12T02:19:08.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.980+0000 7f3c72563640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c691fcd0 0x5555c6922100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.980+0000 7f3c72563640 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c691fcd0 0x5555c6922100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7de1015a86bd7d06 server_cookie=82953a9d9c23db84 in_seq=0 out_seq=0 2024-09-12T02:19:08.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.980+0000 7f3c55ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1053438116 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555c6878860 con 0x5555c691fcd0 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c55ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 client.15555 shutdown 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 msgr2=0x5555c690d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3c64012040 tx=0x7f3c64004a40 comp rx=0 tx=0).stop 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 shutdown_connections 2024-09-12T02:19:08.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6910370 0x5555c690d5d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6906b40 0x5555c690cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c691fcd0 0x5555c6922100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1053438116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c690fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 >> 172.21.15.3:0/1053438116 conn(0x5555c68ce310 msgr2=0x5555c6907960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c55ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 shutdown_connections 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.981+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1053438116 wait complete. 2024-09-12T02:19:08.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.982+0000 7f3c76ba0740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 Processor -- start 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 -- start start 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6901e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6908120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1af810 con 0x5555c6906b40 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1b1350 con 0x5555c6910370 2024-09-12T02:19:08.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1afee0 con 0x5555c69055d0 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c71d62640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c71d62640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45022/0 (socket says 172.21.15.3:45022) 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c71d62640 1 -- 172.21.15.3:0/541726141 learned_addr learned my addr 172.21.15.3:0/541726141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.995+0000 7f3c72563640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6901e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c72d64640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6908120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 -- 172.21.15.3:0/541726141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 msgr2=0x5555c6901e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6901e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 -- 172.21.15.3:0/541726141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 msgr2=0x5555c6908120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6908120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 -- 172.21.15.3:0/541726141 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c690c0d0 con 0x5555c6906b40 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c72d64640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c71d62640 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00bdb0 tx=0x7f3c5c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a76e86cff5059e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:08.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c57fff640 1 -- 172.21.15.3:0/541726141 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c020070 con 0x5555c6906b40 2024-09-12T02:19:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c57fff640 1 -- 172.21.15.3:0/541726141 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c01b8f0 con 0x5555c6906b40 2024-09-12T02:19:08.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.996+0000 7f3c57fff640 1 -- 172.21.15.3:0/541726141 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c010a70 con 0x5555c6906b40 2024-09-12T02:19:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- 172.21.15.3:0/541726141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 msgr2=0x5555c691bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:09.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00bdb0 tx=0x7f3c5c012820 comp rx=0 tx=0).stop 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- 172.21.15.3:0/541726141 shutdown_connections 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6908120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c691bf50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/541726141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6901e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- 172.21.15.3:0/541726141 >> 172.21.15.3:0/541726141 conn(0x5555c68ce310 msgr2=0x5555c6901700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- 172.21.15.3:0/541726141 shutdown_connections 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- 172.21.15.3:0/541726141 wait complete. 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 -- start start 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.997+0000 7f3c76ba0740 1 Processor -- start 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6913e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:09.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6914910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1b0c80 con 0x5555c6906b40 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1b05b0 con 0x5555c6910370 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3c6c1ad7f0 con 0x5555c69055d0 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c76ba0740 10 client.0 _ll_register_callbacks cb 0x7f3c64009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c71d62640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6913e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c71d62640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45026/0 (socket says 172.21.15.3:45026) 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c71d62640 1 -- 172.21.15.3:0/1622904456 learned_addr learned my addr 172.21.15.3:0/1622904456 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6914910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 -- 172.21.15.3:0/1622904456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 msgr2=0x5555c6914910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6914910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 -- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 msgr2=0x5555c69143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 -- 172.21.15.3:0/1622904456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c6989a60 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72d64640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6914910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c72563640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6913e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c68004650 tx=0x7f3c680ef610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d28825af86a71bac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.998+0000 7f3c55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68049040 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f3c680b7870 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c5c01b8f0 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c680bb5e0 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 10 client.25219 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555c698ad40 con 0x5555c69055d0 2024-09-12T02:19:09.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c01b8f0 con 0x5555c69055d0 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6804c060 con 0x5555c69055d0 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 10 client.25219 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 10 client.25219 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 20 client.25219 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c698b210 0x5555c698d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 4 client.25219 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:08.999+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555c6872d70 con 0x5555c698b210 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c698b210 0x5555c698d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c55ffb640 10 client.25219 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c71d62640 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c698b210 0x5555c698d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=993cf7be7e68814b server_cookie=9fa52c1d2b544854 in_seq=0 out_seq=0 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c55ffb640 1 -- 172.21.15.3:0/1622904456 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555c6872d70 con 0x5555c698b210 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c55ffb640 10 client.25219 handle_command_reply: tid=0 2024-09-12T02:19:09.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 client.25219 shutdown 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 msgr2=0x5555c6913e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6913e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c68004650 tx=0x7f3c680ef610 comp rx=0 tx=0).stop 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 shutdown_connections 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555c6910370 0x5555c6914910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555c6906b40 0x5555c69143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5555c698b210 0x5555c698d600 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 --2- 172.21.15.3:0/1622904456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555c69055d0 0x5555c6913e90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c55ffb640 0 client.25219 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.000+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 >> 172.21.15.3:0/1622904456 conn(0x5555c68ce310 msgr2=0x5555c690e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 shutdown_connections 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c76ba0740 1 -- 172.21.15.3:0/1622904456 wait complete. 2024-09-12T02:19:09.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c76ba0740 20 client.25219 trim_cache size 0 max 16384 2024-09-12T02:19:09.008 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:09.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 msgr2=0x7f3c580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 0x7f3c580a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009fd0 tx=0x7f3c64004ab0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 msgr2=0x7f3c6c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3c68006010 tx=0x7f3c680042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 shutdown_connections 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f3c580a2c10 0x7f3c580a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c6c106f10 0x7f3c6c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c6c102790 0x7f3c6c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 --2- 172.21.15.3:0/1982116900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3c6c101de0 0x7f3c6c19a630 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 >> 172.21.15.3:0/1982116900 conn(0x7f3c6c0fd8c0 msgr2=0x7f3c6c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 shutdown_connections 2024-09-12T02:19:09.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:09.001+0000 7f3c747c7640 1 -- 172.21.15.3:0/1982116900 wait complete. 2024-09-12T02:19:09.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:09 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1621583357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:09.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:09 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1621583357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:09.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:09 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1621583357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:09.707 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:19:10.334 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:10 smithi003 ceph-mon[31476]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:10.368 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T02:19:10.368 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T02:19:10.388 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:10.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:10 smithi026 ceph-mon[35496]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:10 smithi092 ceph-mon[36089]: pgmap v359: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 -- 172.21.15.3:0/3210158708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34102d90 msgr2=0x7f0a34103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/3210158708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34102d90 0x7f0a34103170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009cd0 tx=0x7f0a2402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 -- 172.21.15.3:0/3210158708 shutdown_connections 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/3210158708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34108290 0x7f0a3410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/3210158708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/3210158708 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34102d90 0x7f0a34103170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 -- 172.21.15.3:0/3210158708 >> 172.21.15.3:0/3210158708 conn(0x7f0a340fc7d0 msgr2=0x7f0a340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 -- 172.21.15.3:0/3210158708 shutdown_connections 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.885+0000 7f0a38eb3640 1 -- 172.21.15.3:0/3210158708 wait complete. 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 Processor -- start 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 -- start start 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34102d90 0x7f0a341009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 0x7f0a34101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a34112090 con 0x7f0a34108290 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a34111f10 con 0x7f0a34102d90 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a38eb3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a34112210 con 0x7f0a34103740 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 0x7f0a34101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.906 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a31d9b640 1 -- 172.21.15.3:0/643124897 learned_addr learned my addr 172.21.15.3:0/643124897 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.886+0000 7f0a3259c640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34102d90 0x7f0a341009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 -- 172.21.15.3:0/643124897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 msgr2=0x7f0a34100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 -- 172.21.15.3:0/643124897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34102d90 msgr2=0x7f0a341009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34102d90 0x7f0a341009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 -- 172.21.15.3:0/643124897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a341a6a90 con 0x7f0a34108290 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 0x7f0a34101470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d680 tx=0x7f0a2c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a2b7fe640 1 -- 172.21.15.3:0/643124897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c017070 con 0x7f0a34108290 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a341a6d80 con 0x7f0a34108290 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a2b7fe640 1 -- 172.21.15.3:0/643124897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c009820 con 0x7f0a34108290 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a2b7fe640 1 -- 172.21.15.3:0/643124897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c004e00 con 0x7f0a34108290 2024-09-12T02:19:10.907 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.887+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a341a7270 con 0x7f0a34108290 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.890+0000 7f0a2b7fe640 1 -- 172.21.15.3:0/643124897 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0a2c0099d0 con 0x7f0a34108290 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.890+0000 7f0a2b7fe640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 0x7f0a200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3259c640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 0x7f0a200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a2b7fe640 1 -- 172.21.15.3:0/643124897 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0a2c0ce990 con 0x7f0a34108290 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 Processor -- start 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 -- start start 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3259c640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 0x7f0a200a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a24009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a7b40 0x55b5570a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b55700c2f0 con 0x55b5570b1370 2024-09-12T02:19:10.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b556fb3cf0 con 0x55b5570a7b40 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.891+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b5570082b0 con 0x55b5570a65d0 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45054/0 (socket says 172.21.15.3:45054) 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 -- 172.21.15.3:0/1245045810 learned_addr learned my addr 172.21.15.3:0/1245045810 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a3259c640 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a7b40 0x55b5570a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 -- 172.21.15.3:0/1245045810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 msgr2=0x55b5570a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 -- 172.21.15.3:0/1245045810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a7b40 msgr2=0x55b5570a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a7b40 0x55b5570a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 -- 172.21.15.3:0/1245045810 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b556f983a0 con 0x55b5570b1370 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c097590 tx=0x7f0a2c097500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0834de636f7df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a28ff9640 1 -- 172.21.15.3:0/1245045810 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c020070 con 0x55b5570b1370 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a28ff9640 1 -- 172.21.15.3:0/1245045810 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c099430 con 0x55b5570b1370 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.892+0000 7f0a28ff9640 1 -- 172.21.15.3:0/1245045810 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c09a580 con 0x55b5570b1370 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 -- 172.21.15.3:0/1245045810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 msgr2=0x55b5570b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c097590 tx=0x7f0a2c097500 comp rx=0 tx=0).stop 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 -- 172.21.15.3:0/1245045810 shutdown_connections 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b3760 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/1245045810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a7b40 0x55b5570a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 -- 172.21.15.3:0/1245045810 >> 172.21.15.3:0/1245045810 conn(0x55b55706f310 msgr2=0x55b5570a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 -- 172.21.15.3:0/1245045810 shutdown_connections 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.893+0000 7f0a3b28c740 1 -- 172.21.15.3:0/1245045810 wait complete. 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 -- start start 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 Processor -- start 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 0x55b5570b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570b1370 0x55b5570ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b5570b4010 con 0x55b5570a7b40 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b556fc7060 con 0x55b5570a65d0 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b5570b4190 con 0x55b5570b1370 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3b28c740 10 client.0 _ll_register_callbacks cb 0x55b556d1a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a3259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 0x55b5570b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45056/0 (socket says 172.21.15.3:45056) 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a31d9b640 1 -- 172.21.15.3:0/3986959947 learned_addr learned my addr 172.21.15.3:0/3986959947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.894+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570b1370 0x55b5570ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 -- 172.21.15.3:0/3986959947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570b1370 msgr2=0x55b5570ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570b1370 0x55b5570ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 -- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 msgr2=0x55b5570b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 0x55b5570b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 -- 172.21.15.3:0/3986959947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b556d7d3c0 con 0x55b5570a7b40 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a3259c640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 0x55b5570b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:10.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00ba90 tx=0x7f0a1c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f2e044e0ff053 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c01b640 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0a1c01b7e0 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a2c099430 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a1c01b9e0 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a3b28c740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.895+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5570bfbc0 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a2c099430 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a1c019020 con 0x55b5570a7b40 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:10.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 10 client.15573 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 20 client.15573 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b5570c1080 0x55b5570c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 4 client.15573 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b557019860 con 0x55b5570c1080 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.896+0000 7f0a3259c640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b5570c1080 0x55b5570c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.897+0000 7f0a19ffb640 10 client.15573 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.897+0000 7f0a3259c640 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b5570c1080 0x55b5570c3490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bda889541110e79 server_cookie=792d422a2ff2a454 in_seq=0 out_seq=0 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.897+0000 7f0a19ffb640 1 -- 172.21.15.3:0/3986959947 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b557019860 con 0x55b5570c1080 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.897+0000 7f0a19ffb640 10 client.15573 handle_command_reply: tid=0 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.897+0000 7f0a3b28c740 1 client.15573 shutdown 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 msgr2=0x55b5570ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00ba90 tx=0x7f0a1c00bf50 comp rx=0 tx=0).stop 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 shutdown_connections 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570b1370 0x55b5570ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570a7b40 0x55b5570ae030 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b5570c1080 0x55b5570c3490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/3986959947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570a65d0 0x55b5570b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 >> 172.21.15.3:0/3986959947 conn(0x55b55706f310 msgr2=0x55b5570a8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a19ffb640 0 client.15573 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 shutdown_connections 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 1 -- 172.21.15.3:0/3986959947 wait complete. 2024-09-12T02:19:10.915 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.898+0000 7f0a3b28c740 20 client.15573 trim_cache size 0 max 16384 2024-09-12T02:19:10.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.925+0000 7f0a3b28c740 1 Processor -- start 2024-09-12T02:19:10.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.925+0000 7f0a3b28c740 1 -- start start 2024-09-12T02:19:10.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.925+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.925+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.925+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b5570c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341af7f0 con 0x55b5570b1370 2024-09-12T02:19:10.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341b1330 con 0x55b5570c8d60 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341afec0 con 0x55b5570a65d0 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a31d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45062/0 (socket says 172.21.15.3:45062) 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a31d9b640 1 -- 172.21.15.3:0/2811499145 learned_addr learned my addr 172.21.15.3:0/2811499145 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b5570c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 -- 172.21.15.3:0/2811499145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 msgr2=0x55b5570c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b5570c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 -- 172.21.15.3:0/2811499145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 msgr2=0x55b5570bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 -- 172.21.15.3:0/2811499145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5570c3b10 con 0x55b5570a65d0 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:10.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.926+0000 7f0a3259c640 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a2e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a24005e30 tx=0x7f0a2403e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33129b954926f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.927+0000 7f0a1bfff640 1 -- 172.21.15.3:0/2811499145 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24004240 con 0x55b5570a65d0 2024-09-12T02:19:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.927+0000 7f0a1bfff640 1 -- 172.21.15.3:0/2811499145 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a24056070 con 0x55b5570a65d0 2024-09-12T02:19:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.927+0000 7f0a1bfff640 1 -- 172.21.15.3:0/2811499145 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a240305a0 con 0x55b5570a65d0 2024-09-12T02:19:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 -- 172.21.15.3:0/2811499145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 msgr2=0x55b5570a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a2e10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0a24005e30 tx=0x7f0a2403e000 comp rx=0 tx=0).stop 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 -- 172.21.15.3:0/2811499145 shutdown_connections 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b5570c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/2811499145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570a2e10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 -- 172.21.15.3:0/2811499145 >> 172.21.15.3:0/2811499145 conn(0x55b55706f310 msgr2=0x55b5570a2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 -- 172.21.15.3:0/2811499145 shutdown_connections 2024-09-12T02:19:10.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.928+0000 7f0a3b28c740 1 -- 172.21.15.3:0/2811499145 wait complete. 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.929+0000 7f0a3b28c740 1 -- start start 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.929+0000 7f0a3b28c740 1 Processor -- start 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b557133540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341b0c60 con 0x55b5570b1370 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36772/0 (socket says 172.21.15.3:36772) 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3259c640 1 -- 172.21.15.3:0/4015846104 learned_addr learned my addr 172.21.15.3:0/4015846104 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:10.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341b0590 con 0x55b5570c8d60 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a341ad7d0 con 0x55b5570a65d0 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.930+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a32d9d640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b557133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3b28c740 10 client.0 _ll_register_callbacks cb 0x7f0a1c004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 -- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 msgr2=0x55b557133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b557133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 -- 172.21.15.3:0/4015846104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 msgr2=0x55b5570b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b557133a80 con 0x55b5570a65d0 2024-09-12T02:19:10.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a24056070 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a3259c640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a2402b090 tx=0x7f0a24051d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261d0c05618cb2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a19ffb640 10 client.25237 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a19ffb640 1 -- 172.21.15.3:0/4015846104 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2405e070 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.931+0000 7f0a19ffb640 1 -- 172.21.15.3:0/4015846104 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0a24051750 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a19ffb640 1 -- 172.21.15.3:0/4015846104 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a24004000 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 10 client.25237 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b557134ad0 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a19ffb640 1 -- 172.21.15.3:0/4015846104 <== mon.1 v2:172.21.15.92:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a24030070 con 0x55b5570a65d0 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 10 client.25237 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:10.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 10 client.25237 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 20 client.25237 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b557135080 0x55b557137470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 4 client.25237 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b557013d70 con 0x55b557135080 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.932+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b557135080 0x55b557137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a31d9b640 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b557135080 0x55b557137470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4b2260264f6115b server_cookie=298e281d419f1d4e in_seq=0 out_seq=0 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a19ffb640 10 client.25237 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a19ffb640 1 -- 172.21.15.3:0/4015846104 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b557013d70 con 0x55b557135080 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a19ffb640 10 client.25237 handle_command_reply: tid=0 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 client.25237 shutdown 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 msgr2=0x55b5570b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a2402b090 tx=0x7f0a24051d10 comp rx=0 tx=0).stop 2024-09-12T02:19:10.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 shutdown_connections 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55b557135080 0x55b557137470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a19ffb640 0 client.25237 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b5570c8d60 0x55b557133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b5570b1370 0x55b5570b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 --2- 172.21.15.3:0/4015846104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b5570a65d0 0x55b5570b5560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 >> 172.21.15.3:0/4015846104 conn(0x55b55706f310 msgr2=0x55b5570ac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 shutdown_connections 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.933+0000 7f0a3b28c740 1 -- 172.21.15.3:0/4015846104 wait complete. 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a3b28c740 20 client.25237 trim_cache size 0 max 16384 2024-09-12T02:19:10.937 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:10.938 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 msgr2=0x7f0a200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 0x7f0a200a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a24009360 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 msgr2=0x7f0a34101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 0x7f0a34101470 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00d680 tx=0x7f0a2c00db40 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 shutdown_connections 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a200a2c10 0x7f0a200a50d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a34108290 0x7f0a34101470 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34103740 0x7f0a34100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 --2- 172.21.15.3:0/643124897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a34102d90 0x7f0a341009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.934+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 >> 172.21.15.3:0/643124897 conn(0x7f0a340fc7d0 msgr2=0x7f0a34106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.935+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 shutdown_connections 2024-09-12T02:19:10.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:10.935+0000 7f0a38eb3640 1 -- 172.21.15.3:0/643124897 wait complete. 2024-09-12T02:19:10.946 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:11.572 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.087+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2684971882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4106f20 msgr2=0x7fdbb410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.087+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2684971882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4106f20 0x7fdbb410eaa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009ca0 tx=0x7fdbac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2684971882 shutdown_connections 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2684971882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4106f20 0x7fdbb410eaa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2684971882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb41027a0 0x7fdbb41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2684971882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb4101df0 0x7fdbb41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2684971882 >> 172.21.15.3:0/2684971882 conn(0x7fdbb40fd8f0 msgr2=0x7fdbb40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2684971882 shutdown_connections 2024-09-12T02:19:12.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2684971882 wait complete. 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 Processor -- start 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 -- start start 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.088+0000 7fdbbbfc0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbbbfc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb41027a0 0x7fdbb4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbbbfc0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4106f20 0x7fdbb419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbbbfc0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4111f80 con 0x7fdbb4101df0 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbbbfc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4111e00 con 0x7fdbb41027a0 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbbbfc0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4112100 con 0x7fdbb4106f20 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45096/0 (socket says 172.21.15.3:45096) 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/2948757352 learned_addr learned my addr 172.21.15.3:0/2948757352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb955b640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb41027a0 0x7fdbb4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbba55d640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4106f20 0x7fdbb419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/2948757352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4106f20 msgr2=0x7fdbb419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4106f20 0x7fdbb419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/2948757352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb41027a0 msgr2=0x7fdbb4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb41027a0 0x7fdbb4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/2948757352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb419d3b0 con 0x7fdbb4101df0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000bf40 tx=0x7fdbb000dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.089+0000 7fdbaaffd640 1 -- 172.21.15.3:0/2948757352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0018070 con 0x7fdbb4101df0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.090+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb419d6a0 con 0x7fdbb4101df0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.090+0000 7fdbaaffd640 1 -- 172.21.15.3:0/2948757352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb0016070 con 0x7fdbb4101df0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.090+0000 7fdbaaffd640 1 -- 172.21.15.3:0/2948757352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00047a0 con 0x7fdbb4101df0 2024-09-12T02:19:12.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.090+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb4074420 con 0x7fdbb4101df0 2024-09-12T02:19:12.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.092+0000 7fdbaaffd640 1 -- 172.21.15.3:0/2948757352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fdbb0004020 con 0x7fdbb4101df0 2024-09-12T02:19:12.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.093+0000 7fdbbe399740 1 Processor -- start 2024-09-12T02:19:12.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.093+0000 7fdbbe399740 1 -- start start 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567cf72a2a0 con 0x5567cf7c4760 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567cf6d1b10 con 0x5567cf7c5080 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbbe399740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5567cf726260 con 0x5567cf7cf940 2024-09-12T02:19:12.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45104/0 (socket says 172.21.15.3:45104) 2024-09-12T02:19:12.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 -- 172.21.15.3:0/1387731372 learned_addr learned my addr 172.21.15.3:0/1387731372 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:12.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbaaffd640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 0x7fdba00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbb955b640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 -- 172.21.15.3:0/1387731372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 msgr2=0x5567cf7d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 -- 172.21.15.3:0/1387731372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 msgr2=0x5567cf7cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbba55d640 1 -- 172.21.15.3:0/1387731372 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cf49bb90 con 0x5567cf7c4760 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.094+0000 7fdbb955b640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdbaaffd640 1 -- 172.21.15.3:0/2948757352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdbb00d27e0 con 0x7fdbb4101df0 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdbac004af0 tx=0x7fdbac0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22df781eef10fb62 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdbba55d640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 0x7fdba00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdb9ffff640 1 -- 172.21.15.3:0/1387731372 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac04a040 con 0x5567cf7c4760 2024-09-12T02:19:12.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdb9ffff640 1 -- 172.21.15.3:0/1387731372 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac036040 con 0x5567cf7c4760 2024-09-12T02:19:12.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.095+0000 7fdb9ffff640 1 -- 172.21.15.3:0/1387731372 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0047d0 con 0x5567cf7c4760 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.096+0000 7fdbbe399740 1 -- 172.21.15.3:0/1387731372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 msgr2=0x5567cf7c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.096+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdbac004af0 tx=0x7fdbac0098b0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.096+0000 7fdbba55d640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 0x7fdba00a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009a60 tx=0x7fdbac02fa80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 -- 172.21.15.3:0/1387731372 shutdown_connections 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1387731372 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7c4b40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 -- 172.21.15.3:0/1387731372 >> 172.21.15.3:0/1387731372 conn(0x5567cf7c0cf0 msgr2=0x5567cf7c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 -- 172.21.15.3:0/1387731372 shutdown_connections 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 -- 172.21.15.3:0/1387731372 wait complete. 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 -- start start 2024-09-12T02:19:12.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.097+0000 7fdbbe399740 1 Processor -- start 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7cf940 0x5567cf7cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567cf7d23e0 con 0x5567cf7c4760 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567cf6b26e0 con 0x5567cf7cf940 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5567cf7d2560 con 0x5567cf7c5080 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbbe399740 10 client.0 _ll_register_callbacks cb 0x5567cf6d6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:12.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbb9d5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbb955b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbb955b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36816/0 (socket says 172.21.15.3:36816) 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.098+0000 7fdbb955b640 1 -- 172.21.15.3:0/2037972444 learned_addr learned my addr 172.21.15.3:0/2037972444 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbba55d640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7cf940 0x5567cf7cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 -- 172.21.15.3:0/2037972444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7cf940 msgr2=0x5567cf7cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7cf940 0x5567cf7cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 -- 172.21.15.3:0/2037972444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 msgr2=0x5567cf7cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 -- 172.21.15.3:0/2037972444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cf7e4060 con 0x5567cf7c5080 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdbb955b640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b9a0 tx=0x7fdba400be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88baea28abdcc100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.099+0000 7fdb9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4003ea0 con 0x5567cf7c5080 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdba4005220 con 0x5567cf7c5080 2024-09-12T02:19:12.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbac036040 con 0x5567cf7c5080 2024-09-12T02:19:12.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdba4005420 con 0x5567cf7c5080 2024-09-12T02:19:12.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdbbe399740 10 client.25249 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:12.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.100+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567cf7e6a20 con 0x5567cf7c5080 2024-09-12T02:19:12.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac036040 con 0x5567cf7c5080 2024-09-12T02:19:12.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdba4019030 con 0x5567cf7c5080 2024-09-12T02:19:12.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 10 client.25249 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 10 client.25249 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 20 client.25249 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf7e7ee0 0x5567cf7ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 4 client.25249 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567cf7379d0 con 0x5567cf7e7ee0 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.101+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf7e7ee0 0x5567cf7ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.102+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf7e7ee0 0x5567cf7ea2d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf657ec4e35cc00 server_cookie=18c972fc4012cd91 in_seq=0 out_seq=0 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.102+0000 7fdb9d7fa640 10 client.25249 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.102+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/2037972444 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567cf7379d0 con 0x5567cf7e7ee0 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.102+0000 7fdb9d7fa640 10 client.25249 handle_command_reply: tid=0 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.102+0000 7fdbbe399740 1 client.25249 shutdown 2024-09-12T02:19:12.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 msgr2=0x5567cf7cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b9a0 tx=0x7fdba400be60 comp rx=0 tx=0).stop 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 shutdown_connections 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf7e7ee0 0x5567cf7ea2d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7cf940 0x5567cf7cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c5080 0x5567cf7cccc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 --2- 172.21.15.3:0/2037972444 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdb9d7fa640 0 client.25249 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 >> 172.21.15.3:0/2037972444 conn(0x5567cf7c0cf0 msgr2=0x5567cf7bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 shutdown_connections 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 1 -- 172.21.15.3:0/2037972444 wait complete. 2024-09-12T02:19:12.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.103+0000 7fdbbe399740 20 client.25249 trim_cache size 0 max 16384 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.121+0000 7fdbbe399740 1 Processor -- start 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.121+0000 7fdbbe399740 1 -- start start 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c4760 0x5567cf7c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41ab470 con 0x5567cf7cf940 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41acfb0 con 0x5567cf7c5080 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbbe399740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41abb40 con 0x5567cf7c4760 2024-09-12T02:19:12.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45112/0 (socket says 172.21.15.3:45112) 2024-09-12T02:19:12.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 -- 172.21.15.3:0/1254848502 learned_addr learned my addr 172.21.15.3:0/1254848502 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:12.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c4760 0x5567cf7c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbb955b640 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 -- 172.21.15.3:0/1254848502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c4760 msgr2=0x5567cf7c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c4760 0x5567cf7c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 -- 172.21.15.3:0/1254848502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 msgr2=0x5567cf7cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.122+0000 7fdbba55d640 1 -- 172.21.15.3:0/1254848502 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cf7c0b20 con 0x5567cf7cf940 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.123+0000 7fdbba55d640 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005e30 tx=0x7fdbac051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95406bf31bdef1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.123+0000 7fdb9f7fe640 1 -- 172.21.15.3:0/1254848502 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac04a040 con 0x5567cf7cf940 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.123+0000 7fdb9f7fe640 1 -- 172.21.15.3:0/1254848502 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac060070 con 0x5567cf7cf940 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.123+0000 7fdb9f7fe640 1 -- 172.21.15.3:0/1254848502 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0319e0 con 0x5567cf7cf940 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- 172.21.15.3:0/1254848502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 msgr2=0x5567cf7efbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005e30 tx=0x7fdbac051000 comp rx=0 tx=0).stop 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- 172.21.15.3:0/1254848502 shutdown_connections 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7cf940 0x5567cf7efbc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 --2- 172.21.15.3:0/1254848502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7c4760 0x5567cf7c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- 172.21.15.3:0/1254848502 >> 172.21.15.3:0/1254848502 conn(0x5567cf7c0cf0 msgr2=0x5567cf7ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- 172.21.15.3:0/1254848502 shutdown_connections 2024-09-12T02:19:12.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- 172.21.15.3:0/1254848502 wait complete. 2024-09-12T02:19:12.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.124+0000 7fdbbe399740 1 -- start start 2024-09-12T02:19:12.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 Processor -- start 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41ac8e0 con 0x5567cf7c4760 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41ac210 con 0x5567cf7c5080 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41a9450 con 0x5567cf7cf940 2024-09-12T02:19:12.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.125+0000 7fdbbe399740 10 client.0 _ll_register_callbacks cb 0x7fdba4004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45120/0 (socket says 172.21.15.3:45120) 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/843636453 learned_addr learned my addr 172.21.15.3:0/843636453 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb955b640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbba55d640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 msgr2=0x5567cf7db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/843636453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 msgr2=0x5567cf7e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7e0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 -- 172.21.15.3:0/843636453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cf7db540 con 0x5567cf7c4760 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdbb9d5c640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdbb009ba60 tx=0x7fdbb0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72f669675361838 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdb9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:12.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.126+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0021040 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdbb0009c30 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbac060070 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdbb00d03d0 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdbbe399740 10 client.15597 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567cf8532e0 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac060070 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.127+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdbb009e4e0 con 0x5567cf7c4760 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 10 client.15597 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 10 client.15597 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 20 client.15597 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf853a20 0x5567cf855e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 4 client.15597 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T02:19:12.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x5567cf731d20 con 0x5567cf853a20 2024-09-12T02:19:12.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.128+0000 7fdbb955b640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf853a20 0x5567cf855e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:12.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.129+0000 7fdb9d7fa640 10 client.15597 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:12.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.129+0000 7fdbb955b640 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf853a20 0x5567cf855e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868eb82a6d81515a server_cookie=c19a1c27d2f7b0fe in_seq=0 out_seq=0 2024-09-12T02:19:12.141 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.143+0000 7fdb9d7fa640 1 -- 172.21.15.3:0/843636453 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5567cf731d20 con 0x5567cf853a20 2024-09-12T02:19:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.143+0000 7fdb9d7fa640 10 client.15597 handle_command_reply: tid=0 2024-09-12T02:19:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.143+0000 7fdbbe399740 1 client.15597 shutdown 2024-09-12T02:19:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.143+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 msgr2=0x5567cf7e0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.143+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdbb009ba60 tx=0x7fdbb0014040 comp rx=0 tx=0).stop 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 shutdown_connections 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567cf853a20 0x5567cf855e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567cf7cf940 0x5567cf7db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567cf7c5080 0x5567cf7e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 --2- 172.21.15.3:0/843636453 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567cf7c4760 0x5567cf7e0050 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 >> 172.21.15.3:0/843636453 conn(0x5567cf7c0cf0 msgr2=0x5567cf7e2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdb9d7fa640 0 client.15597 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 shutdown_connections 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 1 -- 172.21.15.3:0/843636453 wait complete. 2024-09-12T02:19:12.146 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.144+0000 7fdbbe399740 20 client.15597 trim_cache size 0 max 16384 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 2, 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "ddfbd753": { 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 2, 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:19:12.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.5, 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:12.148 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 msgr2=0x7fdba00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 0x7fdba00a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009a60 tx=0x7fdbac02fa80 comp rx=0 tx=0).stop 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 msgr2=0x7fdbb4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000bf40 tx=0x7fdbb000dd60 comp rx=0 tx=0).stop 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 shutdown_connections 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba00a2c10 0x7fdba00a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbb4106f20 0x7fdbb419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb41027a0 0x7fdbb4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 --2- 172.21.15.3:0/2948757352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb4101df0 0x7fdbb4198400 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:12.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 >> 172.21.15.3:0/2948757352 conn(0x7fdbb40fd8f0 msgr2=0x7fdbb40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:12.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 shutdown_connections 2024-09-12T02:19:12.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:12.145+0000 7fdbbbfc0640 1 -- 172.21.15.3:0/2948757352 wait complete. 2024-09-12T02:19:12.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:12 smithi003 ceph-mon[31476]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T02:19:12.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:12 smithi026 ceph-mon[35496]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T02:19:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:12 smithi092 ceph-mon[36089]: pgmap v360: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T02:19:13.060 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ddfbd753', quiesce took 6.2 seconds. Will release after: 18.3 2024-09-12T02:19:13.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 -- 172.21.15.3:0/2581559389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc102d30 msgr2=0x7efebc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 --2- 172.21.15.3:0/2581559389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc102d30 0x7efebc103110 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009930 tx=0x7efeb802a420 comp rx=0 tx=0).stop 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 -- 172.21.15.3:0/2581559389 shutdown_connections 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 --2- 172.21.15.3:0/2581559389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc1082a0 0x7efebc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 --2- 172.21.15.3:0/2581559389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1036e0 0x7efebc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 --2- 172.21.15.3:0/2581559389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc102d30 0x7efebc103110 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 -- 172.21.15.3:0/2581559389 >> 172.21.15.3:0/2581559389 conn(0x7efebc0fc7d0 msgr2=0x7efebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 -- 172.21.15.3:0/2581559389 shutdown_connections 2024-09-12T02:19:13.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.307+0000 7efec400f640 1 -- 172.21.15.3:0/2581559389 wait complete. 2024-09-12T02:19:13.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.308+0000 7efec400f640 1 Processor -- start 2024-09-12T02:19:13.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.308+0000 7efec400f640 1 -- start start 2024-09-12T02:19:13.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.308+0000 7efec400f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 0x7efebc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.308+0000 7efec400f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.308+0000 7efec400f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 0x7efebc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec400f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1120a0 con 0x7efebc1036e0 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec400f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111f20 con 0x7efebc102d30 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec400f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efebc112220 con 0x7efebc1082a0 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45148/0 (socket says 172.21.15.3:45148) 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 -- 172.21.15.3:0/3343370657 learned_addr learned my addr 172.21.15.3:0/3343370657 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec1dab640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 0x7efebc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec25ac640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 0x7efebc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 -- 172.21.15.3:0/3343370657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 msgr2=0x7efebc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 0x7efebc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 -- 172.21.15.3:0/3343370657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 msgr2=0x7efebc1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 0x7efebc1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 -- 172.21.15.3:0/3343370657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc19d550 con 0x7efebc1036e0 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec25ac640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 0x7efebc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec1dab640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 0x7efebc1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.309+0000 7efec15aa640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9f0 tx=0x7efeac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.310+0000 7efeb2ffd640 1 -- 172.21.15.3:0/3343370657 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00ccc0 con 0x7efebc1036e0 2024-09-12T02:19:13.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.310+0000 7efeb2ffd640 1 -- 172.21.15.3:0/3343370657 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac010910 con 0x7efebc1036e0 2024-09-12T02:19:13.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.310+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc19d840 con 0x7efebc1036e0 2024-09-12T02:19:13.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.310+0000 7efeb2ffd640 1 -- 172.21.15.3:0/3343370657 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac018920 con 0x7efebc1036e0 2024-09-12T02:19:13.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.310+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a5120 con 0x7efebc1036e0 2024-09-12T02:19:13.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.312+0000 7efeb2ffd640 1 -- 172.21.15.3:0/3343370657 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7efeac010ac0 con 0x7efebc1036e0 2024-09-12T02:19:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.313+0000 7efeb2ffd640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 0x7efea80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.313+0000 7efeb2ffd640 1 -- 172.21.15.3:0/3343370657 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efeac0cf500 con 0x7efebc1036e0 2024-09-12T02:19:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.313+0000 7efec1dab640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 0x7efea80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.313+0000 7efec63e8740 1 Processor -- start 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 -- start start 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec1dab640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 0x7efea80a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009900 tx=0x7efeb80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bc9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcadb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55802bc022f0 con 0x55802bc9db40 2024-09-12T02:19:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55802bba9cf0 con 0x55802bc9c5d0 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec63e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55802bbfe2b0 con 0x55802bcab780 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45164/0 (socket says 172.21.15.3:45164) 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec15aa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bc9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.314+0000 7efec1dab640 1 -- 172.21.15.3:0/3792437807 learned_addr learned my addr 172.21.15.3:0/3792437807 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec25ac640 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcadb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 -- 172.21.15.3:0/3792437807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 msgr2=0x55802bcadb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcadb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 -- 172.21.15.3:0/3792437807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 msgr2=0x55802bc9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 -- 172.21.15.3:0/3792437807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802bb8e3a0 con 0x55802bc9c5d0 2024-09-12T02:19:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec1dab640 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efec15aa640 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bc9ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ccea0 tx=0x7efeac09c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fb09d4ce994242 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efea7fff640 1 -- 172.21.15.3:0/3792437807 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0cd0f0 con 0x55802bc9c5d0 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efea7fff640 1 -- 172.21.15.3:0/3792437807 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac086040 con 0x55802bc9c5d0 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.315+0000 7efea7fff640 1 -- 172.21.15.3:0/3792437807 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac09b2a0 con 0x55802bc9c5d0 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 -- 172.21.15.3:0/3792437807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 msgr2=0x55802bc9ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bc9ca10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ccea0 tx=0x7efeac09c430 comp rx=0 tx=0).stop 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 -- 172.21.15.3:0/3792437807 shutdown_connections 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcadb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bc9ca10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 --2- 172.21.15.3:0/3792437807 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9db40 0x55802bc9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.316+0000 7efec63e8740 1 -- 172.21.15.3:0/3792437807 >> 172.21.15.3:0/3792437807 conn(0x55802bc65310 msgr2=0x55802bc98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.317+0000 7efec63e8740 1 -- 172.21.15.3:0/3792437807 shutdown_connections 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.317+0000 7efec63e8740 1 -- 172.21.15.3:0/3792437807 wait complete. 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.317+0000 7efec63e8740 1 -- start start 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.317+0000 7efec63e8740 1 Processor -- start 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bcab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bca8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bcab780 0x55802bca8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55802bcae420 con 0x55802bc9c5d0 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bcab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bcab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45168/0 (socket says 172.21.15.3:45168) 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec1dab640 1 -- 172.21.15.3:0/2783679909 learned_addr learned my addr 172.21.15.3:0/2783679909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55802bbbd060 con 0x55802bcab780 2024-09-12T02:19:13.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55802bcae5a0 con 0x55802bc9db40 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec15aa640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bca8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.318+0000 7efec25ac640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bcab780 0x55802bca8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec63e8740 10 client.0 _ll_register_callbacks cb 0x55802b9108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 -- 172.21.15.3:0/2783679909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bcab780 msgr2=0x55802bca8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bcab780 0x55802bca8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 -- 172.21.15.3:0/2783679909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 msgr2=0x55802bcab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bcab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802b9733c0 con 0x55802bc9db40 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec15aa640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bca8470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efeac099ea0 tx=0x7efeac0cd320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87999bb6cd5234c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:13.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0b8020 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.319+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac086040 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efeac0cf110 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efeac09b040 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 10 client.25267 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55802bcae420 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac086040 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efeac097ad0 con 0x55802bc9db40 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 10 client.25267 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 10 client.25267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 20 client.25267 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bcbd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.320+0000 7efec63e8740 4 client.25267 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:13.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.321+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802bc0f860 con 0x55802bcbb410 2024-09-12T02:19:13.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.321+0000 7efec1dab640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bcbd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.321+0000 7efec1dab640 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bcbd880 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a860b38427f91e1 server_cookie=55e3c8206341d998 in_seq=0 out_seq=0 2024-09-12T02:19:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.321+0000 7efea57fa640 10 client.25267 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.322+0000 7efea57fa640 1 -- 172.21.15.3:0/2783679909 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55802bc0f860 con 0x55802bcbb410 2024-09-12T02:19:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.322+0000 7efea57fa640 10 client.25267 handle_command_reply: tid=0 2024-09-12T02:19:13.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.322+0000 7efec63e8740 1 client.25267 shutdown 2024-09-12T02:19:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.322+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 msgr2=0x55802bca8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.322+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bca8470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efeac099ea0 tx=0x7efeac0cd320 comp rx=0 tx=0).stop 2024-09-12T02:19:13.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 shutdown_connections 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bcab780 0x55802bca8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bcbd880 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bca8470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efea57fa640 0 client.25267 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 --2- 172.21.15.3:0/2783679909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bcab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 >> 172.21.15.3:0/2783679909 conn(0x55802bc65310 msgr2=0x55802bc9e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 shutdown_connections 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 1 -- 172.21.15.3:0/2783679909 wait complete. 2024-09-12T02:19:13.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.323+0000 7efec63e8740 20 client.25267 trim_cache size 0 max 16384 2024-09-12T02:19:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 Processor -- start 2024-09-12T02:19:13.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 -- start start 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9db40 0x55802bcc3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcc36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1ab4a0 con 0x55802bc9c5d0 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1acfe0 con 0x55802bc9db40 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec63e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1abb70 con 0x55802bcab780 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec1dab640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45174/0 (socket says 172.21.15.3:45174) 2024-09-12T02:19:13.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec1dab640 1 -- 172.21.15.3:0/4063370070 learned_addr learned my addr 172.21.15.3:0/4063370070 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:13.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.349+0000 7efec15aa640 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9db40 0x55802bcc3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec25ac640 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcc36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 -- 172.21.15.3:0/4063370070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 msgr2=0x55802bcc36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcc36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 -- 172.21.15.3:0/4063370070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9db40 msgr2=0x55802bcc3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9db40 0x55802bcc3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 -- 172.21.15.3:0/4063370070 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802bc98510 con 0x55802bc9c5d0 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efec1dab640 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009930 tx=0x7efeb803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38777b4f76da2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efea77fe640 1 -- 172.21.15.3:0/4063370070 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8035040 con 0x55802bc9c5d0 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efea77fe640 1 -- 172.21.15.3:0/4063370070 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb8004c50 con 0x55802bc9c5d0 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.350+0000 7efea77fe640 1 -- 172.21.15.3:0/4063370070 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8003aa0 con 0x55802bc9c5d0 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 -- 172.21.15.3:0/4063370070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 msgr2=0x55802bca7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009930 tx=0x7efeb803d000 comp rx=0 tx=0).stop 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 -- 172.21.15.3:0/4063370070 shutdown_connections 2024-09-12T02:19:13.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bcab780 0x55802bcc36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9db40 0x55802bcc3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 --2- 172.21.15.3:0/4063370070 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bc9c5d0 0x55802bca7d30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 -- 172.21.15.3:0/4063370070 >> 172.21.15.3:0/4063370070 conn(0x55802bc65310 msgr2=0x55802bca6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 -- 172.21.15.3:0/4063370070 shutdown_connections 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.351+0000 7efec63e8740 1 -- 172.21.15.3:0/4063370070 wait complete. 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 -- start start 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 Processor -- start 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bcb3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bcaf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1ac910 con 0x55802bcab780 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1ac240 con 0x55802bc9c5d0 2024-09-12T02:19:13.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efebc1a9480 con 0x55802bc9db40 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.352+0000 7efec63e8740 10 client.0 _ll_register_callbacks cb 0x7efeb8040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec25ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec25ac640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45184/0 (socket says 172.21.15.3:45184) 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec25ac640 1 -- 172.21.15.3:0/1970495423 learned_addr learned my addr 172.21.15.3:0/1970495423 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bcb3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec15aa640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bcaf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 -- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 msgr2=0x55802bcaf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bcaf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 -- 172.21.15.3:0/1970495423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 msgr2=0x55802bcaf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 -- 172.21.15.3:0/1970495423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802bcafb70 con 0x55802bc9c5d0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec25ac640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efec1dab640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bcb3d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb80316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3276e3666cbbe631 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efea57fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.353+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8062070 con 0x55802bc9c5d0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb8004c50 con 0x55802bc9c5d0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efeb8004050 con 0x55802bc9c5d0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efeb806e050 con 0x55802bc9c5d0 2024-09-12T02:19:13.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 10 client.25181 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55802bd25f30 con 0x55802bc9c5d0 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeb8004c50 con 0x55802bc9c5d0 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efeb8053450 con 0x55802bc9c5d0 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 10 client.25181 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 10 client.25181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 20 client.25181 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bd28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 4 client.25181 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.354+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55802bc09d70 con 0x55802bcbb410 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efec15aa640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bd28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efec15aa640 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bd28620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d12c0c7b8122353f server_cookie=8ceae83870be65ec in_seq=0 out_seq=0 2024-09-12T02:19:13.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efea57fa640 10 client.25181 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efea57fa640 1 -- 172.21.15.3:0/1970495423 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55802bc09d70 con 0x55802bcbb410 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efea57fa640 10 client.25181 handle_command_reply: tid=0 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.355+0000 7efec63e8740 1 client.25181 shutdown 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 msgr2=0x55802bcb3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bcb3d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb80316b0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 shutdown_connections 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55802bcab780 0x55802bcaf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55802bcbb410 0x55802bd28620 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55802bc9db40 0x55802bcaf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 --2- 172.21.15.3:0/1970495423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55802bc9c5d0 0x55802bcb3d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efea57fa640 0 client.25181 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 >> 172.21.15.3:0/1970495423 conn(0x55802bc65310 msgr2=0x55802bc974e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 shutdown_connections 2024-09-12T02:19:13.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 1 -- 172.21.15.3:0/1970495423 wait complete. 2024-09-12T02:19:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.356+0000 7efec63e8740 20 client.25181 trim_cache size 0 max 16384 2024-09-12T02:19:13.363 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 msgr2=0x7efea80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 0x7efea80a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009900 tx=0x7efeb80058d0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 msgr2=0x7efebc198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efeac00e9f0 tx=0x7efeac00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 shutdown_connections 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efea80a2c90 0x7efea80a5150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc1082a0 0x7efebc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efebc1036e0 0x7efebc198ae0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 --2- 172.21.15.3:0/3343370657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efebc102d30 0x7efebc1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 >> 172.21.15.3:0/3343370657 conn(0x7efebc0fc7d0 msgr2=0x7efebc105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 shutdown_connections 2024-09-12T02:19:13.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:13.357+0000 7efec400f640 1 -- 172.21.15.3:0/3343370657 wait complete. 2024-09-12T02:19:14.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:19:14.426 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:14.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:14 smithi026 ceph-mon[35496]: pgmap v361: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:14.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:14 smithi003 ceph-mon[31476]: pgmap v361: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:14.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:14 smithi092 ceph-mon[36089]: pgmap v361: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 -- 172.21.15.3:0/797381627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c103670 msgr2=0x7f6c4c10bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 --2- 172.21.15.3:0/797381627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c103670 0x7f6c4c10bb60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c3802a480 comp rx=0 tx=0).stop 2024-09-12T02:19:14.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 -- 172.21.15.3:0/797381627 shutdown_connections 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 --2- 172.21.15.3:0/797381627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 0x7f6c4c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 --2- 172.21.15.3:0/797381627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c103670 0x7f6c4c10bb60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 --2- 172.21.15.3:0/797381627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c102d50 0x7f6c4c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 -- 172.21.15.3:0/797381627 >> 172.21.15.3:0/797381627 conn(0x7f6c4c0fc7d0 msgr2=0x7f6c4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 -- 172.21.15.3:0/797381627 shutdown_connections 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.939+0000 7f6c52edc640 1 -- 172.21.15.3:0/797381627 wait complete. 2024-09-12T02:19:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 Processor -- start 2024-09-12T02:19:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 -- start start 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 0x7f6c4c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c19a6d0 0x7f6c4c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c111f80 con 0x7f6c4c102d50 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c111e00 con 0x7f6c4c10c180 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c52edc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112100 con 0x7f6c4c19a6d0 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c50c78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c50c78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45206/0 (socket says 172.21.15.3:45206) 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c50c78640 1 -- 172.21.15.3:0/3014440401 learned_addr learned my addr 172.21.15.3:0/3014440401 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 0x7f6c4c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.940+0000 7f6c51479640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c19a6d0 0x7f6c4c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 -- 172.21.15.3:0/3014440401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c19a6d0 msgr2=0x7f6c4c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c19a6d0 0x7f6c4c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 -- 172.21.15.3:0/3014440401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 msgr2=0x7f6c4c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 0x7f6c4c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 -- 172.21.15.3:0/3014440401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c19b280 con 0x7f6c4c102d50 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000d950 tx=0x7f6c4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c49ffb640 1 -- 172.21.15.3:0/3014440401 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40017070 con 0x7f6c4c102d50 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a73b0 con 0x7f6c4c102d50 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c49ffb640 1 -- 172.21.15.3:0/3014440401 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c40009c30 con 0x7f6c4c102d50 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.941+0000 7f6c49ffb640 1 -- 172.21.15.3:0/3014440401 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000f6b0 con 0x7f6c4c102d50 2024-09-12T02:19:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.942+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a7750 con 0x7f6c4c102d50 2024-09-12T02:19:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.944+0000 7f6c552b5740 1 Processor -- start 2024-09-12T02:19:14.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 -- start start 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f892133090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 0x55f892133a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f8920992f0 con 0x55f8921335d0 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f892040cf0 con 0x55f892134b40 2024-09-12T02:19:14.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c552b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8920952b0 con 0x55f89213e370 2024-09-12T02:19:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c49ffb640 1 -- 172.21.15.3:0/3014440401 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6c400109d0 con 0x7f6c4c102d50 2024-09-12T02:19:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36928/0 (socket says 172.21.15.3:36928) 2024-09-12T02:19:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c4bfff640 1 -- 172.21.15.3:0/705745179 learned_addr learned my addr 172.21.15.3:0/705745179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:14.947 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.945+0000 7f6c50c78640 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 0x55f892133a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 -- 172.21.15.3:0/705745179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 msgr2=0x55f892133090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f892133090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 -- 172.21.15.3:0/705745179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 msgr2=0x55f892133a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 0x55f892133a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 -- 172.21.15.3:0/705745179 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8920253a0 con 0x55f89213e370 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c38005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfc35c6d06fc347 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c50c78640 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 0x55f892133a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c36ffd640 1 -- 172.21.15.3:0/705745179 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38040070 con 0x55f89213e370 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c36ffd640 1 -- 172.21.15.3:0/705745179 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38004030 con 0x55f89213e370 2024-09-12T02:19:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.946+0000 7f6c36ffd640 1 -- 172.21.15.3:0/705745179 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38035040 con 0x55f89213e370 2024-09-12T02:19:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.947+0000 7f6c49ffb640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 0x7f6c3c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.947+0000 7f6c49ffb640 1 -- 172.21.15.3:0/3014440401 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6c400ce9a0 con 0x7f6c4c102d50 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.947+0000 7f6c51479640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 0x7f6c3c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.947+0000 7f6c552b5740 1 -- 172.21.15.3:0/705745179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 msgr2=0x55f892140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.947+0000 7f6c552b5740 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009900 tx=0x7f6c38005d50 comp rx=0 tx=0).stop 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 -- 172.21.15.3:0/705745179 shutdown_connections 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f89213e370 0x55f892140760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f8921335d0 0x55f892133a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 --2- 172.21.15.3:0/705745179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f892133090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 -- 172.21.15.3:0/705745179 >> 172.21.15.3:0/705745179 conn(0x55f8920fc310 msgr2=0x55f89212fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 -- 172.21.15.3:0/705745179 shutdown_connections 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 -- 172.21.15.3:0/705745179 wait complete. 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c51479640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 0x7f6c3c0a5150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005fd0 tx=0x7f6c44004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 -- start start 2024-09-12T02:19:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.948+0000 7f6c552b5740 1 Processor -- start 2024-09-12T02:19:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89213dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f89213b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f892141060 con 0x55f89213e370 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f892054060 con 0x55f892134b40 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f8921411e0 con 0x55f8921335d0 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c552b5740 10 client.0 _ll_register_callbacks cb 0x55f891da78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52580/0 (socket says 172.21.15.3:52580) 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3934736515 learned_addr learned my addr 172.21.15.3:0/3934736515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c51479640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f89213b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.949+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 msgr2=0x55f89213dd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89213dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89213dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3934736515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 msgr2=0x55f89213b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f89213b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3934736515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f891e0a3c0 con 0x55f892134b40 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c51479640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f89213b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005940 tx=0x7f6c38032ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a83e554487af530 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3804c070 con 0x55f892134b40 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c38004030 con 0x55f892134b40 2024-09-12T02:19:14.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.950+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6c38005420 con 0x55f892134b40 2024-09-12T02:19:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c38058050 con 0x55f892134b40 2024-09-12T02:19:14.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 10 client.25190 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f89214cac0 con 0x55f892134b40 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38004030 con 0x55f892134b40 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c38004de0 con 0x55f892134b40 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 10 client.25190 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 10 client.25190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 20 client.25190 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f89214de10 0x55f892150200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 4 client.25190 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.951+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8920a6860 con 0x55f89214de10 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.952+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f89214de10 0x55f892150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.952+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f89214de10 0x55f892150200 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ca9782d017c230f server_cookie=ef81ef7f3920ff85 in_seq=0 out_seq=0 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.952+0000 7f6c2bfff640 10 client.25190 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:14.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c2bfff640 1 -- 172.21.15.3:0/3934736515 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8920a6860 con 0x55f89214de10 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c2bfff640 10 client.25190 handle_command_reply: tid=0 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 client.25190 shutdown 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 msgr2=0x55f89213afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005940 tx=0x7f6c38032ee0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 shutdown_connections 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f89214de10 0x55f892150200 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f89213b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f892134b40 0x55f89213afc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3934736515 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89213dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 >> 172.21.15.3:0/3934736515 conn(0x55f8920fc310 msgr2=0x55f892135930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c2bfff640 0 client.25190 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:14.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 shutdown_connections 2024-09-12T02:19:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 1 -- 172.21.15.3:0/3934736515 wait complete. 2024-09-12T02:19:14.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.953+0000 7f6c552b5740 20 client.25190 trim_cache size 0 max 16384 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 Processor -- start 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 -- start start 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89212fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 0x55f892136120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1afcb0 con 0x55f892134b40 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b17f0 con 0x55f89213e370 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.979+0000 7f6c552b5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b0380 con 0x55f8921335d0 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45220/0 (socket says 172.21.15.3:45220) 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3940755678 learned_addr learned my addr 172.21.15.3:0/3940755678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c51479640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 0x55f892136120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3940755678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 msgr2=0x55f89212fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c50c78640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89212fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89212fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3940755678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 msgr2=0x55f892136120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 0x55f892136120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 -- 172.21.15.3:0/3940755678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f89213d470 con 0x55f892134b40 2024-09-12T02:19:14.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c51479640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 0x55f892136120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c3802a4c0 tx=0x7f6c38005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda2ad648fc4a480 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c367fc640 1 -- 172.21.15.3:0/3940755678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3804c070 con 0x55f892134b40 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.980+0000 7f6c367fc640 1 -- 172.21.15.3:0/3940755678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38003930 con 0x55f892134b40 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c367fc640 1 -- 172.21.15.3:0/3940755678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38040070 con 0x55f892134b40 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 -- 172.21.15.3:0/3940755678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 msgr2=0x55f89213cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c3802a4c0 tx=0x7f6c38005d50 comp rx=0 tx=0).stop 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 -- 172.21.15.3:0/3940755678 shutdown_connections 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f89213e370 0x55f892136120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f892134b40 0x55f89213cf30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 --2- 172.21.15.3:0/3940755678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f8921335d0 0x55f89212fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.981+0000 7f6c552b5740 1 -- 172.21.15.3:0/3940755678 >> 172.21.15.3:0/3940755678 conn(0x55f8920fc310 msgr2=0x55f89213c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.985 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.982+0000 7f6c552b5740 1 -- 172.21.15.3:0/3940755678 shutdown_connections 2024-09-12T02:19:14.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.982+0000 7f6c552b5740 1 -- 172.21.15.3:0/3940755678 wait complete. 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.982+0000 7f6c552b5740 1 -- start start 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.982+0000 7f6c552b5740 1 Processor -- start 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f892134b40 0x55f892150740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f8921b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52600/0 (socket says 172.21.15.3:52600) 2024-09-12T02:19:14.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 -- 172.21.15.3:0/978045820 learned_addr learned my addr 172.21.15.3:0/978045820 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b1120 con 0x55f89213e370 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b0a50 con 0x55f8921335d0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1adc90 con 0x55f892134b40 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f892134b40 0x55f892150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c51479640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f8921b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c552b5740 10 client.0 _ll_register_callbacks cb 0x7f6c38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 -- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f892134b40 msgr2=0x55f892150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f892134b40 0x55f892150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 -- 172.21.15.3:0/978045820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 msgr2=0x55f8921b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f8921b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.983+0000 7f6c50c78640 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8921b6c80 con 0x55f8921335d0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c50c78640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c4009b0d0 tx=0x7f6c40005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46143dc29d75b034 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:14.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40020040 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6c400cc590 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c38003930 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c40009de0 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c552b5740 10 client.25196 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.984+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8921b7e60 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38003930 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c40023060 con 0x55f8921335d0 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 10 client.25196 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 10 client.25196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 20 client.25196 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:14.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f8921b85b0 0x55f8921ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 4 client.25196 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8920a0d70 con 0x55f8921b85b0 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.985+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f8921b85b0 0x55f8921ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c2bfff640 10 client.25196 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c4bfff640 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f8921b85b0 0x55f8921ba9d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ccc574663bbb3aa server_cookie=2ec1b02521208cf4 in_seq=0 out_seq=0 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c2bfff640 1 -- 172.21.15.3:0/978045820 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8920a0d70 con 0x55f8921b85b0 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c2bfff640 10 client.25196 handle_command_reply: tid=0 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 client.25196 shutdown 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 msgr2=0x55f892150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c4009b0d0 tx=0x7f6c40005880 comp rx=0 tx=0).stop 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 shutdown_connections 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55f8921b85b0 0x55f8921ba9d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f89213e370 0x55f8921b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f892134b40 0x55f892150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 --2- 172.21.15.3:0/978045820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f8921335d0 0x55f892150200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 >> 172.21.15.3:0/978045820 conn(0x55f8920fc310 msgr2=0x55f89212eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.986+0000 7f6c2bfff640 0 client.25196 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.987+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 shutdown_connections 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.987+0000 7f6c552b5740 1 -- 172.21.15.3:0/978045820 wait complete. 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.987+0000 7f6c552b5740 20 client.25196 trim_cache size 0 max 16384 2024-09-12T02:19:14.993 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 msgr2=0x7f6c3c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 0x7f6c3c0a5150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005fd0 tx=0x7f6c44004a40 comp rx=0 tx=0).stop 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 msgr2=0x7f6c4c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000d950 tx=0x7f6c4000de10 comp rx=0 tx=0).stop 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 shutdown_connections 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6c3c0a2c90 0x7f6c3c0a5150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c4c19a6d0 0x7f6c4c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c10c180 0x7f6c4c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 --2- 172.21.15.3:0/3014440401 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6c4c102d50 0x7f6c4c1a0600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:14.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 >> 172.21.15.3:0/3014440401 conn(0x7f6c4c0fc7d0 msgr2=0x7f6c4c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:14.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 shutdown_connections 2024-09-12T02:19:14.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:14.988+0000 7f6c52edc640 1 -- 172.21.15.3:0/3014440401 wait complete. 2024-09-12T02:19:15.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:19:16.073 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:16.496 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:16 smithi003 ceph-mon[31476]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:16.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:16 smithi026 ceph-mon[35496]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:16.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:16 smithi092 ceph-mon[36089]: pgmap v362: 97 pgs: 97 active+clean; 664 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 -- 172.21.15.3:0/253424032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848102dd0 msgr2=0x7fb8481031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 --2- 172.21.15.3:0/253424032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848102dd0 0x7fb8481031b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb844009930 tx=0x7fb84402a420 comp rx=0 tx=0).stop 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 -- 172.21.15.3:0/253424032 shutdown_connections 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 --2- 172.21.15.3:0/253424032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848108300 0x7fb84810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 --2- 172.21.15.3:0/253424032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848103780 0x7fb848107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 --2- 172.21.15.3:0/253424032 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848102dd0 0x7fb8481031b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 -- 172.21.15.3:0/253424032 >> 172.21.15.3:0/253424032 conn(0x7fb8480fc7d0 msgr2=0x7fb8480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 -- 172.21.15.3:0/253424032 shutdown_connections 2024-09-12T02:19:16.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.589+0000 7fb84fb45640 1 -- 172.21.15.3:0/253424032 wait complete. 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 Processor -- start 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 -- start start 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848102dd0 0x7fb848100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848108300 0x7fb8481014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112090 con 0x7fb848103780 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb848111f10 con 0x7fb848102dd0 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84fb45640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb848112210 con 0x7fb848108300 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84d0e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84d0e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45252/0 (socket says 172.21.15.3:45252) 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84d0e0640 1 -- 172.21.15.3:0/1347735044 learned_addr learned my addr 172.21.15.3:0/1347735044 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:16.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.590+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848102dd0 0x7fb848100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848108300 0x7fb8481014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 -- 172.21.15.3:0/1347735044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848108300 msgr2=0x7fb8481014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848108300 0x7fb8481014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 -- 172.21.15.3:0/1347735044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848102dd0 msgr2=0x7fb848100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848102dd0 0x7fb848100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 -- 172.21.15.3:0/1347735044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8481a6b10 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb83800d740 tx=0x7fb83800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb83effd640 1 -- 172.21.15.3:0/1347735044 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb83effd640 1 -- 172.21.15.3:0/1347735044 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83800de50 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8481a6e00 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb83effd640 1 -- 172.21.15.3:0/1347735044 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83800f6b0 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.591+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a7300 con 0x7fb848103780 2024-09-12T02:19:16.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.594+0000 7fb83effd640 1 -- 172.21.15.3:0/1347735044 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb83801a070 con 0x7fb848103780 2024-09-12T02:19:16.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 Processor -- start 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 -- start start 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cea3c092f0 con 0x55cea3cb6b80 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cea3bb0cf0 con 0x55cea3cad350 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.595+0000 7fb851f1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cea3c052b0 con 0x55cea3cabde0 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb83effd640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 0x7fb8340a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52632/0 (socket says 172.21.15.3:52632) 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 -- 172.21.15.3:0/281895770 learned_addr learned my addr 172.21.15.3:0/281895770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 0x7fb8340a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 -- 172.21.15.3:0/281895770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 msgr2=0x55cea3cac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 -- 172.21.15.3:0/281895770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 msgr2=0x55cea3cb8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84e0e2640 1 -- 172.21.15.3:0/281895770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea3b953a0 con 0x55cea3cad350 2024-09-12T02:19:16.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb83effd640 1 -- 172.21.15.3:0/1347735044 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8380cebc0 con 0x7fb848103780 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.596+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.597+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb840014940 tx=0x7fb840014d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866401c785d3ef90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.597+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 0x7fb8340a51d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb84001c000 tx=0x7fb84001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.597+0000 7fb833fff640 1 -- 172.21.15.3:0/281895770 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84000c930 con 0x55cea3cad350 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.597+0000 7fb833fff640 1 -- 172.21.15.3:0/281895770 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb84001e3f0 con 0x55cea3cad350 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.597+0000 7fb833fff640 1 -- 172.21.15.3:0/281895770 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84000f5c0 con 0x55cea3cad350 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- 172.21.15.3:0/281895770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 msgr2=0x55cea3cab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb840014940 tx=0x7fb840014d40 comp rx=0 tx=0).stop 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- 172.21.15.3:0/281895770 shutdown_connections 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 --2- 172.21.15.3:0/281895770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cab8a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- 172.21.15.3:0/281895770 >> 172.21.15.3:0/281895770 conn(0x55cea3c6c310 msgr2=0x55cea3ca8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- 172.21.15.3:0/281895770 shutdown_connections 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- 172.21.15.3:0/281895770 wait complete. 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 -- start start 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.598+0000 7fb851f1e740 1 Processor -- start 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cb3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cea3cb97c0 con 0x55cea3cb6b80 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cea3bc4060 con 0x55cea3cad350 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cea3cb9940 con 0x55cea3cabde0 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb851f1e740 10 client.0 _ll_register_callbacks cb 0x55cea39178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:16.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84e0e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84d8e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84e0e2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45266/0 (socket says 172.21.15.3:45266) 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84d0e0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cb3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84e0e2640 1 -- 172.21.15.3:0/3921672614 learned_addr learned my addr 172.21.15.3:0/3921672614 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.599+0000 7fb84d0e0640 1 -- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 msgr2=0x55cea3cb6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84d0e0640 1 -- 172.21.15.3:0/3921672614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 msgr2=0x55cea3cb3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84d0e0640 1 -- 172.21.15.3:0/3921672614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea397a3c0 con 0x55cea3cad350 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cb3830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8380cc500 tx=0x7fb8380975b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae9a0deccdab0dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb8317fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:16.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838021070 con 0x55cea3cad350 2024-09-12T02:19:16.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8380bb2b0 con 0x55cea3cad350 2024-09-12T02:19:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb84001e3f0 con 0x55cea3cad350 2024-09-12T02:19:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb838009c30 con 0x55cea3cad350 2024-09-12T02:19:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb851f1e740 10 client.25208 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:16.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cea3cc52b0 con 0x55cea3cad350 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.600+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83809b950 con 0x55cea3cad350 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb83809f100 con 0x55cea3cad350 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 10 client.25208 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 10 client.25208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 20 client.25208 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3cc6600 0x55cea3cc8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 4 client.25208 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb84001e3f0 con 0x55cea3cc6600 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3cc6600 0x55cea3cc8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb8317fa640 10 client.25208 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.601+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3cc6600 0x55cea3cc8a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39c76c1bc0303a1 server_cookie=8ca8ac99f9170e57 in_seq=0 out_seq=0 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb8317fa640 1 -- 172.21.15.3:0/3921672614 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fb84001e3f0 con 0x55cea3cc6600 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb8317fa640 10 client.25208 handle_command_reply: tid=0 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 client.25208 shutdown 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 msgr2=0x55cea3cb3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cb3830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8380cc500 tx=0x7fb8380975b0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 shutdown_connections 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3cc6600 0x55cea3cc8a20 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cb3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cb3830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.602+0000 7fb851f1e740 1 --2- 172.21.15.3:0/3921672614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.603+0000 7fb8317fa640 0 client.25208 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.603+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 >> 172.21.15.3:0/3921672614 conn(0x55cea3c6c310 msgr2=0x55cea3cae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.603+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 shutdown_connections 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.603+0000 7fb851f1e740 1 -- 172.21.15.3:0/3921672614 wait complete. 2024-09-12T02:19:16.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.603+0000 7fb851f1e740 20 client.25208 trim_cache size 0 max 16384 2024-09-12T02:19:16.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 Processor -- start 2024-09-12T02:19:16.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 -- start start 2024-09-12T02:19:16.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cb6b80 0x55cea3caee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806dfe0 con 0x55cea3cad350 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806fb20 con 0x55cea3cb6b80 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb851f1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806e6b0 con 0x55cea3cabde0 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb84d0e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb84d0e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45272/0 (socket says 172.21.15.3:45272) 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.616+0000 7fb84d0e0640 1 -- 172.21.15.3:0/4246958822 learned_addr learned my addr 172.21.15.3:0/4246958822 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cb6b80 0x55cea3caee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 -- 172.21.15.3:0/4246958822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 msgr2=0x55cea3cb2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 -- 172.21.15.3:0/4246958822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cb6b80 msgr2=0x55cea3caee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cb6b80 0x55cea3caee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 -- 172.21.15.3:0/4246958822 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea3ca9170 con 0x55cea3cad350 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb838009610 tx=0x7fb838097620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1b2af9bad9d1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb8337fe640 1 -- 172.21.15.3:0/4246958822 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838021070 con 0x55cea3cad350 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb8337fe640 1 -- 172.21.15.3:0/4246958822 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83809b950 con 0x55cea3cad350 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.617+0000 7fb8337fe640 1 -- 172.21.15.3:0/4246958822 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380bb310 con 0x55cea3cad350 2024-09-12T02:19:16.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 -- 172.21.15.3:0/4246958822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 msgr2=0x55cea3cae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb838009610 tx=0x7fb838097620 comp rx=0 tx=0).stop 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 -- 172.21.15.3:0/4246958822 shutdown_connections 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cb6b80 0x55cea3caee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cad350 0x55cea3cae930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4246958822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cb2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 -- 172.21.15.3:0/4246958822 >> 172.21.15.3:0/4246958822 conn(0x55cea3c6c310 msgr2=0x55cea3c9ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 -- 172.21.15.3:0/4246958822 shutdown_connections 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.618+0000 7fb851f1e740 1 -- 172.21.15.3:0/4246958822 wait complete. 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.619+0000 7fb851f1e740 1 -- start start 2024-09-12T02:19:16.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.619+0000 7fb851f1e740 1 Processor -- start 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.619+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.619+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.619+0000 7fb851f1e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cbbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb851f1e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806f450 con 0x55cea3cb6b80 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb851f1e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806ed80 con 0x55cea3cad350 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb851f1e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb84806bfe0 con 0x55cea3cabde0 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb851f1e740 10 client.0 _ll_register_callbacks cb 0x7fb838097a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36994/0 (socket says 172.21.15.3:36994) 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 -- 172.21.15.3:0/4250052475 learned_addr learned my addr 172.21.15.3:0/4250052475 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:16.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84e0e2640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cbbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 -- 172.21.15.3:0/4250052475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 msgr2=0x55cea3cbf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cbf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 -- 172.21.15.3:0/4250052475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 msgr2=0x55cea3cbbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cbbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 -- 172.21.15.3:0/4250052475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea3cbc0e0 con 0x55cea3cabde0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb84d8e1640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8440096c0 tx=0x7fb844031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f60d638279ce59bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.620+0000 7fb8317fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844035040 con 0x55cea3cabde0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb844005020 con 0x55cea3cabde0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83809b950 con 0x55cea3cabde0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb844005220 con 0x55cea3cabde0 2024-09-12T02:19:16.628 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb851f1e740 10 client.25303 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.621+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cea3d31930 con 0x55cea3cabde0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb83809b950 con 0x55cea3cabde0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb8440035f0 con 0x55cea3cabde0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 10 client.25303 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 10 client.25303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 20 client.25303 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3d320b0 0x55cea3d344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 4 client.25303 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.622+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cea3c16860 con 0x55cea3d320b0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3d320b0 0x55cea3d344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb8317fa640 10 client.25303 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb84d0e0640 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3d320b0 0x55cea3d344d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f32364236491ef8 server_cookie=6dd2fc0d35f93ba7 in_seq=0 out_seq=0 2024-09-12T02:19:16.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb8317fa640 1 -- 172.21.15.3:0/4250052475 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cea3c16860 con 0x55cea3d320b0 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb8317fa640 10 client.25303 handle_command_reply: tid=0 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 client.25303 shutdown 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 msgr2=0x55cea3cbebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8440096c0 tx=0x7fb844031b10 comp rx=0 tx=0).stop 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 shutdown_connections 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cea3d320b0 0x55cea3d344d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb8317fa640 0 client.25303 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cea3cb6b80 0x55cea3cbbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cea3cad350 0x55cea3cbf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 --2- 172.21.15.3:0/4250052475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cea3cabde0 0x55cea3cbebd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.623+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 >> 172.21.15.3:0/4250052475 conn(0x55cea3c6c310 msgr2=0x55cea3cb5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 shutdown_connections 2024-09-12T02:19:16.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb851f1e740 1 -- 172.21.15.3:0/4250052475 wait complete. 2024-09-12T02:19:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb851f1e740 20 client.25303 trim_cache size 0 max 16384 2024-09-12T02:19:16.631 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:16.631 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 msgr2=0x7fb8340a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 0x7fb8340a51d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb84001c000 tx=0x7fb84001e000 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 msgr2=0x7fb848100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.624+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb83800d740 tx=0x7fb83800dc00 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 shutdown_connections 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8340a2d10 0x7fb8340a51d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb848108300 0x7fb8481014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb848103780 0x7fb848100f70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 --2- 172.21.15.3:0/1347735044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848102dd0 0x7fb848100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 >> 172.21.15.3:0/1347735044 conn(0x7fb8480fc7d0 msgr2=0x7fb848105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 shutdown_connections 2024-09-12T02:19:16.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:16.625+0000 7fb84fb45640 1 -- 172.21.15.3:0/1347735044 wait complete. 2024-09-12T02:19:18.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:19:18.264 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:19:18.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:18 smithi026 ceph-mon[35496]: pgmap v363: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:18.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:18 smithi003 ceph-mon[31476]: pgmap v363: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:18.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:18 smithi092 ceph-mon[36089]: pgmap v363: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:18.570 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 -- 172.21.15.3:0/3988162100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 msgr2=0x7fce981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/3988162100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce981067e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fce88009900 tx=0x7fce8802a410 comp rx=0 tx=0).stop 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 -- 172.21.15.3:0/3988162100 shutdown_connections 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/3988162100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98106f10 0x7fce9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/3988162100 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce981067e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/3988162100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98101de0 0x7fce981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 -- 172.21.15.3:0/3988162100 >> 172.21.15.3:0/3988162100 conn(0x7fce980fd8e0 msgr2=0x7fce980ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 -- 172.21.15.3:0/3988162100 shutdown_connections 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.321+0000 7fce9f6e6640 1 -- 172.21.15.3:0/3988162100 wait complete. 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 Processor -- start 2024-09-12T02:19:19.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 -- start start 2024-09-12T02:19:19.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98101de0 0x7fce98198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98106f10 0x7fce9819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce98111f70 con 0x7fce98102790 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce98111df0 con 0x7fce98101de0 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9f6e6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce981120f0 con 0x7fce98106f10 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9cc81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9cc81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36884/0 (socket says 172.21.15.3:36884) 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9cc81640 1 -- 172.21.15.3:0/1481741816 learned_addr learned my addr 172.21.15.3:0/1481741816 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9d482640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98101de0 0x7fce98198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.322+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98106f10 0x7fce9819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 -- 172.21.15.3:0/1481741816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98106f10 msgr2=0x7fce9819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98106f10 0x7fce9819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 -- 172.21.15.3:0/1481741816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98101de0 msgr2=0x7fce98198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98101de0 0x7fce98198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 -- 172.21.15.3:0/1481741816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9819d400 con 0x7fce98102790 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce88006010 tx=0x7fce880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce8e7fc640 1 -- 172.21.15.3:0/1481741816 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8803f030 con 0x7fce98102790 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9819d690 con 0x7fce98102790 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce8e7fc640 1 -- 172.21.15.3:0/1481741816 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce88003ce0 con 0x7fce98102790 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce8e7fc640 1 -- 172.21.15.3:0/1481741816 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88035070 con 0x7fce98102790 2024-09-12T02:19:19.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.323+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce98074420 con 0x7fce98102790 2024-09-12T02:19:19.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fce8e7fc640 1 -- 172.21.15.3:0/1481741816 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fce88042030 con 0x7fce98102790 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 Processor -- start 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 -- start start 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac904b60 0x55a1ac904f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac912130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac86a280 con 0x55a1ac905480 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac811b10 con 0x55a1ac90fd40 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.327+0000 7fcea1abf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac866240 con 0x55a1ac904b60 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9dc83640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac904b60 0x55a1ac904f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36900/0 (socket says 172.21.15.3:36900) 2024-09-12T02:19:19.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 -- 172.21.15.3:0/1342535880 learned_addr learned my addr 172.21.15.3:0/1342535880 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:19.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac912130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 -- 172.21.15.3:0/1342535880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac904b60 msgr2=0x55a1ac904f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac904b60 0x55a1ac904f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce8e7fc640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 0x7fce840ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 -- 172.21.15.3:0/1342535880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 msgr2=0x55a1ac912130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac912130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 -- 172.21.15.3:0/1342535880 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ac81e1f0 con 0x55a1ac905480 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce8e7fc640 1 -- 172.21.15.3:0/1481741816 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fce880ef1a0 con 0x7fce98102790 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.328+0000 7fce9d482640 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce94009d20 tx=0x7fce94011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266d45a08637ec9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.329+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 0x7fce840ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.329+0000 7fce837fe640 1 -- 172.21.15.3:0/1342535880 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9401e070 con 0x55a1ac905480 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.329+0000 7fce837fe640 1 -- 172.21.15.3:0/1342535880 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce94019b10 con 0x55a1ac905480 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.329+0000 7fce837fe640 1 -- 172.21.15.3:0/1342535880 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9400ebf0 con 0x55a1ac905480 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- 172.21.15.3:0/1342535880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 msgr2=0x55a1ac90f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce94009d20 tx=0x7fce94011810 comp rx=0 tx=0).stop 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 0x7fce840ad960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce90005fd0 tx=0x7fce90009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- 172.21.15.3:0/1342535880 shutdown_connections 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac912130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac90f800 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1342535880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac904b60 0x55a1ac904f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- 172.21.15.3:0/1342535880 >> 172.21.15.3:0/1342535880 conn(0x55a1ac9010f0 msgr2=0x55a1ac9014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- 172.21.15.3:0/1342535880 shutdown_connections 2024-09-12T02:19:19.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- 172.21.15.3:0/1342535880 wait complete. 2024-09-12T02:19:19.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.330+0000 7fcea1abf740 1 -- start start 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 Processor -- start 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac90d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac912910 con 0x55a1ac904b60 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac824f10 con 0x55a1ac90fd40 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a1ac912a90 con 0x55a1ac905480 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fcea1abf740 10 client.0 _ll_register_callbacks cb 0x55a1ac7eda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fce9d482640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fce9d482640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36912/0 (socket says 172.21.15.3:36912) 2024-09-12T02:19:19.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.331+0000 7fce9d482640 1 -- 172.21.15.3:0/3505038055 learned_addr learned my addr 172.21.15.3:0/3505038055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9dc83640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9cc81640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac90d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 -- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 msgr2=0x55a1ac90d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac90d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 -- 172.21.15.3:0/3505038055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 msgr2=0x55a1ac90d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 -- 172.21.15.3:0/3505038055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ac5db6a0 con 0x55a1ac904b60 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9cc81640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac90d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9dc83640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce9d482640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fce94011d10 tx=0x7fce94019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dce32dc41db298 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9401e070 con 0x55a1ac904b60 2024-09-12T02:19:19.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce94019b10 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.332+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fce9400eaf0 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce940098f0 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 10 client.15660 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1ac91e540 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce94019b10 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce940047d0 con 0x55a1ac904b60 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 20 client.15660 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac91f950 0x55a1ac921d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 4 client.15660 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.333+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ac8777f0 con 0x55a1ac91f950 2024-09-12T02:19:19.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.334+0000 7fce9cc81640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac91f950 0x55a1ac921d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.334+0000 7fce80ff9640 10 client.15660 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.334+0000 7fce9cc81640 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac91f950 0x55a1ac921d40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa2ad2951a62e3 server_cookie=eb42681a7c7b6a9d in_seq=0 out_seq=0 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fce80ff9640 1 -- 172.21.15.3:0/3505038055 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1ac8777f0 con 0x55a1ac91f950 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fce80ff9640 10 client.15660 handle_command_reply: tid=0 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 client.15660 shutdown 2024-09-12T02:19:19.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 msgr2=0x55a1ac90cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fce94011d10 tx=0x7fce94019660 comp rx=0 tx=0).stop 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 shutdown_connections 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac91f950 0x55a1ac921d40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac90d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 --2- 172.21.15.3:0/3505038055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac904b60 0x55a1ac90cc50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fce80ff9640 0 client.15660 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 >> 172.21.15.3:0/3505038055 conn(0x55a1ac9010f0 msgr2=0x55a1ac906dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 shutdown_connections 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.335+0000 7fcea1abf740 1 -- 172.21.15.3:0/3505038055 wait complete. 2024-09-12T02:19:19.340 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.336+0000 7fcea1abf740 20 client.15660 trim_cache size 0 max 16384 2024-09-12T02:19:19.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 Processor -- start 2024-09-12T02:19:19.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 -- start start 2024-09-12T02:19:19.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac9075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 0x55a1ac927930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce981ab320 con 0x55a1ac90fd40 2024-09-12T02:19:19.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce981ace60 con 0x55a1ac9274d0 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fcea1abf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce981ab9f0 con 0x55a1ac905480 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fce9cc81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fce9cc81640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36914/0 (socket says 172.21.15.3:36914) 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.353+0000 7fce9cc81640 1 -- 172.21.15.3:0/1261242125 learned_addr learned my addr 172.21.15.3:0/1261242125 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9d482640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac9075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 0x55a1ac927930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 -- 172.21.15.3:0/1261242125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 msgr2=0x55a1ac9075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac9075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 -- 172.21.15.3:0/1261242125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 msgr2=0x55a1ac927930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 0x55a1ac927930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 -- 172.21.15.3:0/1261242125 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ac9222a0 con 0x55a1ac90fd40 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 0x55a1ac927930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:19.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fce88000c00 tx=0x7fce880a4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501f7b95c558d425 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce82ffd640 1 -- 172.21.15.3:0/1261242125 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88049040 con 0x55a1ac90fd40 2024-09-12T02:19:19.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce82ffd640 1 -- 172.21.15.3:0/1261242125 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce880f2b40 con 0x55a1ac90fd40 2024-09-12T02:19:19.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.354+0000 7fce82ffd640 1 -- 172.21.15.3:0/1261242125 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880bba00 con 0x55a1ac90fd40 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 -- 172.21.15.3:0/1261242125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 msgr2=0x55a1ac90d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fce88000c00 tx=0x7fce880a4f90 comp rx=0 tx=0).stop 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 -- 172.21.15.3:0/1261242125 shutdown_connections 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac9274d0 0x55a1ac927930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac90fd40 0x55a1ac90d520 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1261242125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac905480 0x55a1ac9075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.355+0000 7fcea1abf740 1 -- 172.21.15.3:0/1261242125 >> 172.21.15.3:0/1261242125 conn(0x55a1ac9010f0 msgr2=0x55a1ac900270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.356+0000 7fcea1abf740 1 -- 172.21.15.3:0/1261242125 shutdown_connections 2024-09-12T02:19:19.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.356+0000 7fcea1abf740 1 -- 172.21.15.3:0/1261242125 wait complete. 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.356+0000 7fcea1abf740 1 -- start start 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.356+0000 7fcea1abf740 1 Processor -- start 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac914610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 0x55a1ac914b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce981ac790 con 0x55a1ac905480 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fce981ac0c0 con 0x55a1ac982460 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fce981a9300 con 0x55a1ac9274d0 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fcea1abf740 10 client.0 _ll_register_callbacks cb 0x7fce9401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fce9dc83640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fce9dc83640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35110/0 (socket says 172.21.15.3:35110) 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.357+0000 7fce9dc83640 1 -- 172.21.15.3:0/1964764633 learned_addr learned my addr 172.21.15.3:0/1964764633 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:19.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9d482640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac914610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 0x55a1ac914b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 -- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 msgr2=0x55a1ac914b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 0x55a1ac914b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 -- 172.21.15.3:0/1964764633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 msgr2=0x55a1ac914610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac914610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 -- 172.21.15.3:0/1964764633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ac983020 con 0x55a1ac982460 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9cc81640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 0x55a1ac914b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce9dc83640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce90009e20 tx=0x7fce90004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=285fdf3cbaf329b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:19.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:19.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9001f810 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce880f2b40 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fce9001f9b0 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.358+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce9001fbb0 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 10 client.25229 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1ac98b3d0 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce880f2b40 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce90034050 con 0x55a1ac982460 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 10 client.25229 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 10 client.25229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 20 client.25229 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac98bad0 0x55a1ac98def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:19.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 4 client.25229 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.359+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1ac871d00 con 0x55a1ac98bad0 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.360+0000 7fce9d482640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac98bad0 0x55a1ac98def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.360+0000 7fce9d482640 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac98bad0 0x55a1ac98def0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61c1d892f3dd3e5d server_cookie=e9e327d2d90e3e0a in_seq=0 out_seq=0 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.360+0000 7fce80ff9640 10 client.25229 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.361+0000 7fce80ff9640 1 -- 172.21.15.3:0/1964764633 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1ac871d00 con 0x55a1ac98bad0 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.361+0000 7fce80ff9640 10 client.25229 handle_command_reply: tid=0 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.361+0000 7fcea1abf740 1 client.25229 shutdown 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 msgr2=0x55a1ac9828e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fce90009e20 tx=0x7fce90004a40 comp rx=0 tx=0).stop 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 shutdown_connections 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1ac98bad0 0x55a1ac98def0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1ac982460 0x55a1ac9828e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1ac9274d0 0x55a1ac914b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 --2- 172.21.15.3:0/1964764633 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1ac905480 0x55a1ac914610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 >> 172.21.15.3:0/1964764633 conn(0x55a1ac9010f0 msgr2=0x55a1ac926ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fce80ff9640 0 client.25229 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 shutdown_connections 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 1 -- 172.21.15.3:0/1964764633 wait complete. 2024-09-12T02:19:19.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.362+0000 7fcea1abf740 20 client.25229 trim_cache size 0 max 16384 2024-09-12T02:19:19.366 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-12T02:19:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "ba0922e5-605c-45a8-a78d-01da890a692a", 2024-09-12T02:19:19.366 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-12T02:19:19.366 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 msgr2=0x7fce840ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 0x7fce840ad960 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce90005fd0 tx=0x7fce90009600 comp rx=0 tx=0).stop 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 msgr2=0x7fce98198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce88006010 tx=0x7fce880042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 shutdown_connections 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fce840ab4a0 0x7fce840ad960 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce98106f10 0x7fce9819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce98102790 0x7fce98198990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 --2- 172.21.15.3:0/1481741816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce98101de0 0x7fce98198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:19.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 >> 172.21.15.3:0/1481741816 conn(0x7fce980fd8e0 msgr2=0x7fce980febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:19.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 shutdown_connections 2024-09-12T02:19:19.368 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:19.363+0000 7fce9f6e6640 1 -- 172.21.15.3:0/1481741816 wait complete. 2024-09-12T02:19:20.035 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:19:20.319 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:20.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:20 smithi003 ceph-mon[31476]: pgmap v364: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:19:20.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:20 smithi003 ceph-mon[31476]: scrub queued for path: / 2024-09-12T02:19:20.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:20 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:20 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:20.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:20 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:20 smithi026 ceph-mon[35496]: pgmap v364: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:19:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:20 smithi026 ceph-mon[35496]: scrub queued for path: / 2024-09-12T02:19:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:20 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:20 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:20.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:20 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:20 smithi092 ceph-mon[36089]: pgmap v364: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:19:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:20 smithi092 ceph-mon[36089]: scrub queued for path: / 2024-09-12T02:19:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:20 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:20 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:20 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:20.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.853+0000 7f0a1997a640 1 -- 172.21.15.3:0/3739820923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14101410 msgr2=0x7f0a1410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.853+0000 7f0a1997a640 1 --2- 172.21.15.3:0/3739820923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14101410 0x7f0a1410d940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009c10 tx=0x7f0a0c02a6f0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 -- 172.21.15.3:0/3739820923 shutdown_connections 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 --2- 172.21.15.3:0/3739820923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14101410 0x7f0a1410d940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 --2- 172.21.15.3:0/3739820923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14100a70 0x7f0a14100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 --2- 172.21.15.3:0/3739820923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 0x7f0a14106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 -- 172.21.15.3:0/3739820923 >> 172.21.15.3:0/3739820923 conn(0x7f0a140fc790 msgr2=0x7f0a140febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.854+0000 7f0a1997a640 1 -- 172.21.15.3:0/3739820923 shutdown_connections 2024-09-12T02:19:20.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.855+0000 7f0a1997a640 1 -- 172.21.15.3:0/3739820923 wait complete. 2024-09-12T02:19:20.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.855+0000 7f0a1997a640 1 Processor -- start 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.855+0000 7f0a1997a640 1 -- start start 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14100a70 0x7f0a14102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 0x7f0a141030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14110e20 con 0x7f0a14106a70 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14110ca0 con 0x7f0a14101410 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a1997a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a14110fa0 con 0x7f0a14100a70 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35134/0 (socket says 172.21.15.3:35134) 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a0bfff640 1 -- 172.21.15.3:0/351077086 learned_addr learned my addr 172.21.15.3:0/351077086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.856+0000 7f0a137fe640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 0x7f0a141030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 -- 172.21.15.3:0/351077086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14100a70 msgr2=0x7f0a14102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14100a70 0x7f0a14102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 -- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 msgr2=0x7f0a141030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 0x7f0a141030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 -- 172.21.15.3:0/351077086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a141a68c0 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a18978640 1 -- 172.21.15.3:0/351077086 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00ccc0 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a141a6bb0 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a18978640 1 -- 172.21.15.3:0/351077086 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc005500 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a141a70f0 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.857+0000 7f0a18978640 1 -- 172.21.15.3:0/351077086 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010690 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a18978640 1 -- 172.21.15.3:0/351077086 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f09fc0046c0 con 0x7f0a14101410 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 Processor -- start 2024-09-12T02:19:20.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 -- start start 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94384090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438ee90 0x560e94391280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560e942e12e0 con 0x560e943845d0 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e94288ce0 con 0x560e94383cb0 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.861+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e942dd2a0 con 0x560e9438ee90 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438ee90 0x560e94391280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36952/0 (socket says 172.21.15.3:36952) 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a12ffd640 1 -- 172.21.15.3:0/1761429578 learned_addr learned my addr 172.21.15.3:0/1761429578 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a137fe640 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94384090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.862+0000 7f0a18978640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 0x7f0a000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a18978640 1 -- 172.21.15.3:0/351077086 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f09fc0cf5a0 con 0x7f0a14101410 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a137fe640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 0x7f0a000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 -- 172.21.15.3:0/1761429578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438ee90 msgr2=0x560e94391280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438ee90 0x560e94391280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 -- 172.21.15.3:0/1761429578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 msgr2=0x560e94384090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94384090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 -- 172.21.15.3:0/1761429578 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9426d440 con 0x560e943845d0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.863+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009ae0 tx=0x7f0a04009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b978957551591fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.864+0000 7f0a09ffb640 1 -- 172.21.15.3:0/1761429578 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400ecb0 con 0x560e943845d0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.864+0000 7f0a09ffb640 1 -- 172.21.15.3:0/1761429578 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a0400ee50 con 0x560e943845d0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 -- 172.21.15.3:0/1761429578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 msgr2=0x560e9438e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009ae0 tx=0x7f0a04009ee0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a137fe640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 0x7f0a000a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c004af0 tx=0x7f0a0c02a6f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 -- 172.21.15.3:0/1761429578 shutdown_connections 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438ee90 0x560e94391280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e943845d0 0x560e9438e950 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/1761429578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94384090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 -- 172.21.15.3:0/1761429578 >> 172.21.15.3:0/1761429578 conn(0x560e94380240 msgr2=0x560e94380620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 -- 172.21.15.3:0/1761429578 shutdown_connections 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.865+0000 7f0a1bd53740 1 -- 172.21.15.3:0/1761429578 wait complete. 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 -- start start 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 Processor -- start 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9438c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560e94391b00 con 0x560e94383cb0 2024-09-12T02:19:20.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560e9429c050 con 0x560e9438ee90 2024-09-12T02:19:20.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.866+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560e94391c80 con 0x560e9438bd50 2024-09-12T02:19:20.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a1bd53740 10 client.0 _ll_register_callbacks cb 0x560e942dc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36956/0 (socket says 172.21.15.3:36956) 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a12ffd640 1 -- 172.21.15.3:0/2737868742 learned_addr learned my addr 172.21.15.3:0/2737868742 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35162/0 (socket says 172.21.15.3:35162) 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a137fe640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9438c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 -- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 msgr2=0x560e9438c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9438c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 -- 172.21.15.3:0/2737868742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 msgr2=0x560e9438e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a0bfff640 1 -- 172.21.15.3:0/2737868742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e940523c0 con 0x560e9438ee90 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.867+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0ccee0 tx=0x7f09fc098000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=147c6c619770a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f09f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc09bc30 con 0x560e9438ee90 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f09fc0a4410 con 0x560e9438ee90 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e942ee850 con 0x560e9438ee90 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09fc09ffe0 con 0x560e9438ee90 2024-09-12T02:19:20.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f0a1bd53740 10 client.25241 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:20.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.868+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e9439d700 con 0x560e9438ee90 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.869+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560e942ee850 con 0x560e9438ee90 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f09fc09b080 con 0x560e9438ee90 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 10 client.25241 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 10 client.25241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 20 client.25241 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e9439ec10 0x560e943a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 4 client.25241 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e942e8d60 con 0x560e9439ec10 2024-09-12T02:19:20.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.870+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e9439ec10 0x560e943a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.871+0000 7f09f2ffd640 10 client.25241 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.871+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e9439ec10 0x560e943a1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f38d42b7b33b56 server_cookie=bccfb8fb5fb6fa10 in_seq=0 out_seq=0 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.871+0000 7f09f2ffd640 1 -- 172.21.15.3:0/2737868742 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e942e8d60 con 0x560e9439ec10 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.871+0000 7f09f2ffd640 10 client.25241 handle_command_reply: tid=0 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.871+0000 7f0a1bd53740 1 client.25241 shutdown 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 msgr2=0x560e9438b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09fc0ccee0 tx=0x7f09fc098000 comp rx=0 tx=0).stop 2024-09-12T02:19:20.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 shutdown_connections 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e9439ec10 0x560e943a1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9438c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e9438b810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/2737868742 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e9438e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f09f2ffd640 0 client.25241 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 >> 172.21.15.3:0/2737868742 conn(0x560e94380240 msgr2=0x560e94385780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 shutdown_connections 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 1 -- 172.21.15.3:0/2737868742 wait complete. 2024-09-12T02:19:20.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.872+0000 7f0a1bd53740 20 client.25241 trim_cache size 0 max 16384 2024-09-12T02:19:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 Processor -- start 2024-09-12T02:19:20.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 -- start start 2024-09-12T02:19:20.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9439adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e94386d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141ab440 con 0x560e94383cb0 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141acf80 con 0x560e9438ee90 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141abb10 con 0x560e9438bd50 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36962/0 (socket says 172.21.15.3:36962) 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a12ffd640 1 -- 172.21.15.3:0/3992937539 learned_addr learned my addr 172.21.15.3:0/3992937539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.907+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9439adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a137fe640 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e94386d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 -- 172.21.15.3:0/3992937539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 msgr2=0x560e9439adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9439adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 -- 172.21.15.3:0/3992937539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 msgr2=0x560e94386d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e94386d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 -- 172.21.15.3:0/3992937539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e94389d60 con 0x560e94383cb0 2024-09-12T02:19:20.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.908+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72424ca3f463d0da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.909+0000 7f0a097fa640 1 -- 172.21.15.3:0/3992937539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04009370 con 0x560e94383cb0 2024-09-12T02:19:20.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.909+0000 7f0a097fa640 1 -- 172.21.15.3:0/3992937539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a04032070 con 0x560e94383cb0 2024-09-12T02:19:20.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.909+0000 7f0a097fa640 1 -- 172.21.15.3:0/3992937539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04010d70 con 0x560e94383cb0 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- 172.21.15.3:0/3992937539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 msgr2=0x560e94380620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04005fc0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- 172.21.15.3:0/3992937539 shutdown_connections 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e9438ee90 0x560e94386d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e9439adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/3992937539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e94383cb0 0x560e94380620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- 172.21.15.3:0/3992937539 >> 172.21.15.3:0/3992937539 conn(0x560e94380240 msgr2=0x560e94386680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- 172.21.15.3:0/3992937539 shutdown_connections 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- 172.21.15.3:0/3992937539 wait complete. 2024-09-12T02:19:20.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.910+0000 7f0a1bd53740 1 -- start start 2024-09-12T02:19:20.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 Processor -- start 2024-09-12T02:19:20.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e94388a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e9438ee90 0x560e94388fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141ac8b0 con 0x560e9438ee90 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141ac1e0 con 0x560e94383cb0 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0a141a94a0 con 0x560e9438bd50 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a1bd53740 10 client.0 _ll_register_callbacks cb 0x7f09fc0d2b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35188/0 (socket says 172.21.15.3:35188) 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a12ffd640 1 -- 172.21.15.3:0/845941250 learned_addr learned my addr 172.21.15.3:0/845941250 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.911+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e94388a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 -- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 msgr2=0x560e94388a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e94388a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 -- 172.21.15.3:0/845941250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e9438ee90 msgr2=0x560e94388fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e9438ee90 0x560e94388fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.919 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 -- 172.21.15.3:0/845941250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e94389510 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a12ffd640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a040098c0 tx=0x7f0a04005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53fd4a3ce4360962 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f09f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0403a070 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0a04010b80 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.912+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a04032070 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0a040093f0 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f0a1bd53740 10 client.25247 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e94390610 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a04004520 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0a0403d030 con 0x560e94383cb0 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f0a1bd53740 10 client.25247 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:20.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f0a1bd53740 10 client.25247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.913+0000 7f0a1bd53740 20 client.25247 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e94409d00 0x560e9440c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f0a1bd53740 4 client.25247 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0a04032070 con 0x560e94409d00 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e94409d00 0x560e9440c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f09f2ffd640 10 client.25247 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.914+0000 7f0a0bfff640 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e94409d00 0x560e9440c0f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa3377d1896617a server_cookie=39aeb544934331c4 in_seq=0 out_seq=0 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f09f2ffd640 1 -- 172.21.15.3:0/845941250 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f0a04032070 con 0x560e94409d00 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f09f2ffd640 10 client.25247 handle_command_reply: tid=0 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 client.25247 shutdown 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 msgr2=0x560e94397370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0a040098c0 tx=0x7f0a04005ab0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 shutdown_connections 2024-09-12T02:19:20.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560e94409d00 0x560e9440c0f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e9438ee90 0x560e94388fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e9438bd50 0x560e94388a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 --2- 172.21.15.3:0/845941250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560e94383cb0 0x560e94397370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 >> 172.21.15.3:0/845941250 conn(0x560e94380240 msgr2=0x560e9438a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f09f2ffd640 0 client.25247 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 shutdown_connections 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 1 -- 172.21.15.3:0/845941250 wait complete. 2024-09-12T02:19:20.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.915+0000 7f0a1bd53740 20 client.25247 trim_cache size 0 max 16384 2024-09-12T02:19:20.923 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-12T02:19:20.923 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-12T02:19:20.923 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:20.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 msgr2=0x7f0a000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 0x7f0a000a5150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c004af0 tx=0x7f0a0c02a6f0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 msgr2=0x7f0a14102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 shutdown_connections 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0a000a2c90 0x7f0a000a5150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a14106a70 0x7f0a141030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a14101410 0x7f0a14102b80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 --2- 172.21.15.3:0/351077086 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a14100a70 0x7f0a14102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 >> 172.21.15.3:0/351077086 conn(0x7f0a140fc790 msgr2=0x7f0a141048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 shutdown_connections 2024-09-12T02:19:20.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:20.916+0000 7f0a1997a640 1 -- 172.21.15.3:0/351077086 wait complete. 2024-09-12T02:19:21.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:21 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:19:21.529 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:21 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:19:21.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:21 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.751+0000 7f01a40c9640 1 -- 172.21.15.3:0/2001247803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 msgr2=0x7f019c108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.751+0000 7f01a40c9640 1 --2- 172.21.15.3:0/2001247803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c108e70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f018c009900 tx=0x7f018c02a410 comp rx=0 tx=0).stop 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 -- 172.21.15.3:0/2001247803 shutdown_connections 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 --2- 172.21.15.3:0/2001247803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c108e70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 --2- 172.21.15.3:0/2001247803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 0x7f019c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 --2- 172.21.15.3:0/2001247803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 0x7f019c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 -- 172.21.15.3:0/2001247803 >> 172.21.15.3:0/2001247803 conn(0x7f019c0fc190 msgr2=0x7f019c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 -- 172.21.15.3:0/2001247803 shutdown_connections 2024-09-12T02:19:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 -- 172.21.15.3:0/2001247803 wait complete. 2024-09-12T02:19:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 Processor -- start 2024-09-12T02:19:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.752+0000 7f01a40c9640 1 -- start start 2024-09-12T02:19:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 0x7f019c196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 0x7f019c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f019c10c6e0 con 0x7f019c1025e0 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f019c10c560 con 0x7f019c101c20 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a40c9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f019c10c860 con 0x7f019c100a30 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36990/0 (socket says 172.21.15.3:36990) 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 -- 172.21.15.3:0/1382630621 learned_addr learned my addr 172.21.15.3:0/1382630621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 0x7f019c196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a1664640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 0x7f019c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 -- 172.21.15.3:0/1382630621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 msgr2=0x7f019c196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 0x7f019c196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 -- 172.21.15.3:0/1382630621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 msgr2=0x7f019c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 0x7f019c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.753+0000 7f01a2666640 1 -- 172.21.15.3:0/1382630621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c19e8b0 con 0x7f019c1025e0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f01a2666640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a930 tx=0x7f018c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f0192ffd640 1 -- 172.21.15.3:0/1382630621 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c046070 con 0x7f019c1025e0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c06a230 con 0x7f019c1025e0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c06a770 con 0x7f019c1025e0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f0192ffd640 1 -- 172.21.15.3:0/1382630621 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f018c003ad0 con 0x7f019c1025e0 2024-09-12T02:19:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.754+0000 7f0192ffd640 1 -- 172.21.15.3:0/1382630621 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035050 con 0x7f019c1025e0 2024-09-12T02:19:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.757+0000 7f0192ffd640 1 -- 172.21.15.3:0/1382630621 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f018c049070 con 0x7f019c1025e0 2024-09-12T02:19:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.757+0000 7f01a6e62740 1 Processor -- start 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.757+0000 7f01a6e62740 1 -- start start 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a35582cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582d2e0 0x55a355831da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a3557902f0 con 0x55a355832370 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a355738270 con 0x55a35582c9a0 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a6e62740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a35578c2b0 con 0x55a35582d2e0 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37000/0 (socket says 172.21.15.3:37000) 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 -- 172.21.15.3:0/3345413205 learned_addr learned my addr 172.21.15.3:0/3345413205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1e65640 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582d2e0 0x55a355831da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a35582cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 -- 172.21.15.3:0/3345413205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582d2e0 msgr2=0x55a355831da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582d2e0 0x55a355831da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 -- 172.21.15.3:0/3345413205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 msgr2=0x55a35582cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a35582cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f01a1664640 1 -- 172.21.15.3:0/3345413205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a355737340 con 0x55a355832370 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.758+0000 7f0192ffd640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 0x7f01880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.759+0000 7f0192ffd640 1 -- 172.21.15.3:0/1382630621 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f018c0f0a90 con 0x7f019c1025e0 2024-09-12T02:19:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.759+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 0x7f01880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.759+0000 7f01a1664640 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f90 tx=0x7f0198005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454d3cb48e6b1d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.760+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 0x7f01880a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a35582dd10 tx=0x7f0194007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.760+0000 7f0187fff640 1 -- 172.21.15.3:0/3345413205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198004050 con 0x55a355832370 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.760+0000 7f0187fff640 1 -- 172.21.15.3:0/3345413205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01980041f0 con 0x55a355832370 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.760+0000 7f0187fff640 1 -- 172.21.15.3:0/3345413205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019801d740 con 0x55a355832370 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- 172.21.15.3:0/3345413205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 msgr2=0x55a355834830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f90 tx=0x7f0198005850 comp rx=0 tx=0).stop 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- 172.21.15.3:0/3345413205 shutdown_connections 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a355834830 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582d2e0 0x55a355831da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 --2- 172.21.15.3:0/3345413205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a35582cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- 172.21.15.3:0/3345413205 >> 172.21.15.3:0/3345413205 conn(0x55a355828010 msgr2=0x55a355828410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- 172.21.15.3:0/3345413205 shutdown_connections 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- 172.21.15.3:0/3345413205 wait complete. 2024-09-12T02:19:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.761+0000 7f01a6e62740 1 -- start start 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 Processor -- start 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a355836c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582d2e0 0x55a355837190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a355835340 con 0x55a355832370 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a355719290 con 0x55a35582d2e0 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a3558354c0 con 0x55a35582c9a0 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a6e62740 10 client.0 _ll_register_callbacks cb 0x55a35574a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a2666640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a2666640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37016/0 (socket says 172.21.15.3:37016) 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a2666640 1 -- 172.21.15.3:0/2423076894 learned_addr learned my addr 172.21.15.3:0/2423076894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a1e65640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a355836c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.762+0000 7f01a1664640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582d2e0 0x55a355837190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 -- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 msgr2=0x55a355836c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a355836c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 -- 172.21.15.3:0/2423076894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582d2e0 msgr2=0x55a355837190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582d2e0 0x55a355837190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 -- 172.21.15.3:0/2423076894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3554a3210 con 0x55a355832370 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a2666640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f018c0b9c00 tx=0x7f018c0bcc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd56ca8364cb6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01857fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0c5df0 con 0x55a355832370 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f018c0dd0c0 con 0x55a355832370 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01980041f0 con 0x55a355832370 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f018c003c80 con 0x55a355832370 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a6e62740 10 client.15693 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.763+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a35556f620 con 0x55a355832370 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01980041f0 con 0x55a355832370 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f018c0c5960 con 0x55a355832370 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 10 client.15693 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 20 client.15693 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558429e0 0x55a355844e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 4 client.15693 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a35579da20 con 0x55a3558429e0 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.764+0000 7f01a1e65640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558429e0 0x55a355844e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01857fa640 10 client.15693 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01a1e65640 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558429e0 0x55a355844e40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f737ee1295021891 server_cookie=1726a788f8e7e200 in_seq=0 out_seq=0 2024-09-12T02:19:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01857fa640 1 -- 172.21.15.3:0/2423076894 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a35579da20 con 0x55a3558429e0 2024-09-12T02:19:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01857fa640 10 client.15693 handle_command_reply: tid=0 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01a6e62740 1 client.15693 shutdown 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 msgr2=0x55a35583e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.765+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f018c0b9c00 tx=0x7f018c0bcc50 comp rx=0 tx=0).stop 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 shutdown_connections 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558429e0 0x55a355844e40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a355832370 0x55a35583e1d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01857fa640 0 client.15693 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582d2e0 0x55a355837190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 --2- 172.21.15.3:0/2423076894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a355836c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 >> 172.21.15.3:0/2423076894 conn(0x55a355828010 msgr2=0x55a355829820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 shutdown_connections 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 1 -- 172.21.15.3:0/2423076894 wait complete. 2024-09-12T02:19:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.766+0000 7f01a6e62740 20 client.15693 trim_cache size 0 max 16384 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.794+0000 7f01a6e62740 1 Processor -- start 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.794+0000 7f01a6e62740 1 -- start start 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.794+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a355828ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 0x55a355829400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a6e62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f018000e4b0 con 0x55a35582d2e0 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a6e62740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f018000fff0 con 0x55a35582c9a0 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a6e62740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f018000eb80 con 0x55a355832370 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37032/0 (socket says 172.21.15.3:37032) 2024-09-12T02:19:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 -- 172.21.15.3:0/1847116274 learned_addr learned my addr 172.21.15.3:0/1847116274 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a355828ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a2666640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 0x55a355829400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 -- 172.21.15.3:0/1847116274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 msgr2=0x55a355829400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 0x55a355829400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 -- 172.21.15.3:0/1847116274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 msgr2=0x55a355828ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a355828ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 -- 172.21.15.3:0/1847116274 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a355827e30 con 0x55a35582d2e0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a2666640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 0x55a355829400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01a1664640 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f90 tx=0x7f0198001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3772f103d88f7355 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.795+0000 7f01877fe640 1 -- 172.21.15.3:0/1847116274 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198005600 con 0x55a35582d2e0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.796+0000 7f01877fe640 1 -- 172.21.15.3:0/1847116274 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0198003ad0 con 0x55a35582d2e0 2024-09-12T02:19:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.796+0000 7f01877fe640 1 -- 172.21.15.3:0/1847116274 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0198027910 con 0x55a35582d2e0 2024-09-12T02:19:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.796+0000 7f01a6e62740 1 -- 172.21.15.3:0/1847116274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 msgr2=0x55a35583e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.796+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0198009f90 tx=0x7f0198001e50 comp rx=0 tx=0).stop 2024-09-12T02:19:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 -- 172.21.15.3:0/1847116274 shutdown_connections 2024-09-12T02:19:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a355832370 0x55a355829400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583e190 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1847116274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a35582c9a0 0x55a355828ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 -- 172.21.15.3:0/1847116274 >> 172.21.15.3:0/1847116274 conn(0x55a355828010 msgr2=0x55a355836c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 -- 172.21.15.3:0/1847116274 shutdown_connections 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 -- 172.21.15.3:0/1847116274 wait complete. 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 -- start start 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.797+0000 7f01a6e62740 1 Processor -- start 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a35583b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a355832370 0x55a35583ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f018000f920 con 0x55a35582d2e0 2024-09-12T02:19:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f018000f250 con 0x55a355832370 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f018000c490 con 0x55a35582c9a0 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a6e62740 10 client.0 _ll_register_callbacks cb 0x7f018c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37034/0 (socket says 172.21.15.3:37034) 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 -- 172.21.15.3:0/1795830275 learned_addr learned my addr 172.21.15.3:0/1795830275 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a35583b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a2666640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a355832370 0x55a35583ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 -- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 msgr2=0x55a35583b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a35583b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 -- 172.21.15.3:0/1795830275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a355832370 msgr2=0x55a35583ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a355832370 0x55a35583ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1664640 1 -- 172.21.15.3:0/1795830275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a35583d4c0 con 0x55a35582d2e0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.798+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a35583b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a1664640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0198006010 tx=0x7f01980274b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14caac57b8f30a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01980035f0 con 0x55a35582d2e0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0198003790 con 0x55a35582d2e0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0198003ad0 con 0x55a35582d2e0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01980276d0 con 0x55a35582d2e0 2024-09-12T02:19:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3558ad800 con 0x55a35582d2e0 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0198003ad0 con 0x55a35582d2e0 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f019801e710 con 0x55a35582d2e0 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.799+0000 7f01a6e62740 20 client.15699 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558adf10 0x55a3558b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01a6e62740 4 client.15699 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3558320c0 con 0x55a3558adf10 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558adf10 0x55a3558b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01857fa640 10 client.15699 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.800+0000 7f01a1e65640 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558adf10 0x55a3558b0330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b72abb7bd064000 server_cookie=70b6550de23c3a5 in_seq=0 out_seq=0 2024-09-12T02:19:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.807+0000 7f01857fa640 1 -- 172.21.15.3:0/1795830275 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3558320c0 con 0x55a3558adf10 2024-09-12T02:19:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.807+0000 7f01857fa640 10 client.15699 handle_command_reply: tid=0 2024-09-12T02:19:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.807+0000 7f01a6e62740 1 client.15699 shutdown 2024-09-12T02:19:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 msgr2=0x55a35583bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0198006010 tx=0x7f01980274b0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 shutdown_connections 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a3558adf10 0x55a3558b0330 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a355832370 0x55a35583ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01857fa640 0 client.15699 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a35582d2e0 0x55a35583bea0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 --2- 172.21.15.3:0/1795830275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a35582c9a0 0x55a35583b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 >> 172.21.15.3:0/1795830275 conn(0x55a355828010 msgr2=0x55a3558269a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 shutdown_connections 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 1 -- 172.21.15.3:0/1795830275 wait complete. 2024-09-12T02:19:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.808+0000 7f01a6e62740 20 client.15699 trim_cache size 0 max 16384 2024-09-12T02:19:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:19:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:19:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:19:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:19:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 msgr2=0x7f01880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 0x7f01880a5150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55a35582dd10 tx=0x7f0194007600 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 msgr2=0x7f019c19e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a930 tx=0x7f018c032c60 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 shutdown_connections 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f01880a2c90 0x7f01880a5150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f019c1025e0 0x7f019c19e210 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f019c101c20 0x7f019c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 --2- 172.21.15.3:0/1382630621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f019c100a30 0x7f019c196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 >> 172.21.15.3:0/1382630621 conn(0x7f019c0fc190 msgr2=0x7f019c104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 shutdown_connections 2024-09-12T02:19:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:21.809+0000 7f01a40c9640 1 -- 172.21.15.3:0/1382630621 wait complete. 2024-09-12T02:19:21.943 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:19:21.943 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:19:22.313 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:22 smithi026 ceph-mon[35496]: pgmap v365: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T02:19:22.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:22 smithi003 ceph-mon[31476]: pgmap v365: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T02:19:22.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:22 smithi092 ceph-mon[36089]: pgmap v365: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T02:19:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.831+0000 7f48acc55640 1 -- 172.21.15.3:0/148959802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a8102d30 msgr2=0x7f48a8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.831+0000 7f48acc55640 1 --2- 172.21.15.3:0/148959802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a8102d30 0x7f48a8103110 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4898009930 tx=0x7f489802a420 comp rx=0 tx=0).stop 2024-09-12T02:19:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 -- 172.21.15.3:0/148959802 shutdown_connections 2024-09-12T02:19:22.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 --2- 172.21.15.3:0/148959802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 0x7f48a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 --2- 172.21.15.3:0/148959802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a81036e0 0x7f48a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 --2- 172.21.15.3:0/148959802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a8102d30 0x7f48a8103110 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 -- 172.21.15.3:0/148959802 >> 172.21.15.3:0/148959802 conn(0x7f48a80fc7d0 msgr2=0x7f48a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 -- 172.21.15.3:0/148959802 shutdown_connections 2024-09-12T02:19:22.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.832+0000 7f48acc55640 1 -- 172.21.15.3:0/148959802 wait complete. 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 Processor -- start 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 -- start start 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 0x7f48a81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 0x7f48a819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:22.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48acc55640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8112100 con 0x7f48a81036e0 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48a5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48a5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37042/0 (socket says 172.21.15.3:37042) 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.833+0000 7f48a5d9b640 1 -- 172.21.15.3:0/3496672752 learned_addr learned my addr 172.21.15.3:0/3496672752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8111f80 con 0x7f48a81082a0 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8112280 con 0x7f48a8102d30 2024-09-12T02:19:22.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a6d9d640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 0x7f48a819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a659c640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 0x7f48a81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 -- 172.21.15.3:0/3496672752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 msgr2=0x7f48a81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 0x7f48a81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 -- 172.21.15.3:0/3496672752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 msgr2=0x7f48a819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 0x7f48a819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819d570 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a659c640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 0x7f48a81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.834+0000 7f48a5d9b640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d750 tx=0x7f489c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.835+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c017070 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.835+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489c005500 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.835+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c004e00 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.835+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819d860 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.835+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a5140 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.837+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f489c004fa0 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.838+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a806a850 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.838+0000 7f48977fe640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 0x7f48900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.838+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f489c0ce7b0 con 0x7f48a81036e0 2024-09-12T02:19:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.838+0000 7f48a659c640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 0x7f48900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.839+0000 7f48a659c640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 0x7f48900a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f489802a940 tx=0x7f4898005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:22.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:22.847+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f489c09b5a0 con 0x7f48a81036e0 2024-09-12T02:19:23.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.261+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48a810a2b0 con 0x7f48a81036e0 2024-09-12T02:19:23.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.263+0000 7f48977fe640 1 -- 172.21.15.3:0/3496672752 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f489c09aea0 con 0x7f48a81036e0 2024-09-12T02:19:23.264 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:19:23.264 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:19:23.264 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.265+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 msgr2=0x7f48900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.265+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 0x7f48900a5150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f489802a940 tx=0x7f4898005fd0 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.265+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 msgr2=0x7f48a8198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.265+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d750 tx=0x7f489c00dc10 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 shutdown_connections 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f48900a2c90 0x7f48900a5150 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48a81082a0 0x7f48a819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f48a81036e0 0x7f48a8198b00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 --2- 172.21.15.3:0/3496672752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48a8102d30 0x7f48a81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 >> 172.21.15.3:0/3496672752 conn(0x7f48a80fc7d0 msgr2=0x7f48a8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 shutdown_connections 2024-09-12T02:19:23.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:23.266+0000 7f48acc55640 1 -- 172.21.15.3:0/3496672752 wait complete. 2024-09-12T02:19:24.115 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:19:24.514 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:24.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:24 smithi026 ceph-mon[35496]: pgmap v366: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T02:19:24.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:24 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3496672752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:24 smithi003 ceph-mon[31476]: pgmap v366: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T02:19:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:24 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3496672752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:24 smithi092 ceph-mon[36089]: pgmap v366: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T02:19:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:24 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3496672752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 -- 172.21.15.3:0/4105756345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c105560 msgr2=0x7f021c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 --2- 172.21.15.3:0/4105756345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c105560 0x7f021c10d0e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0214009ca0 tx=0x7f021402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 -- 172.21.15.3:0/4105756345 shutdown_connections 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 --2- 172.21.15.3:0/4105756345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c105560 0x7f021c10d0e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 --2- 172.21.15.3:0/4105756345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c100de0 0x7f021c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 --2- 172.21.15.3:0/4105756345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.068+0000 7f0223a7c640 1 -- 172.21.15.3:0/4105756345 >> 172.21.15.3:0/4105756345 conn(0x7f021c077230 msgr2=0x7f021c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.069+0000 7f0223a7c640 1 -- 172.21.15.3:0/4105756345 shutdown_connections 2024-09-12T02:19:25.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.069+0000 7f0223a7c640 1 -- 172.21.15.3:0/4105756345 wait complete. 2024-09-12T02:19:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 Processor -- start 2024-09-12T02:19:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 -- start start 2024-09-12T02:19:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c100de0 0x7f021c10ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c105560 0x7f021c10f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0221017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c100de0 0x7f021c10ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111f40 con 0x7f021c100de0 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111dc0 con 0x7f021c100430 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0223a7c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1120c0 con 0x7f021c105560 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35268/0 (socket says 172.21.15.3:35268) 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.070+0000 7f0221818640 1 -- 172.21.15.3:0/3449649994 learned_addr learned my addr 172.21.15.3:0/3449649994 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0222019640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c105560 0x7f021c10f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 -- 172.21.15.3:0/3449649994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c105560 msgr2=0x7f021c10f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c105560 0x7f021c10f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 -- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c100de0 msgr2=0x7f021c10ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c100de0 0x7f021c10ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 -- 172.21.15.3:0/3449649994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c1a6880 con 0x7f021c100430 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0221818640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f021800bf20 tx=0x7f021800dd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0212ffd640 1 -- 172.21.15.3:0/3449649994 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021801c070 con 0x7f021c100430 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c198180 con 0x7f021c100430 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0212ffd640 1 -- 172.21.15.3:0/3449649994 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0218009d00 con 0x7f021c100430 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.071+0000 7f0212ffd640 1 -- 172.21.15.3:0/3449649994 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021801fd50 con 0x7f021c100430 2024-09-12T02:19:25.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.072+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1986c0 con 0x7f021c100430 2024-09-12T02:19:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.075+0000 7f0212ffd640 1 -- 172.21.15.3:0/3449649994 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0218029440 con 0x7f021c100430 2024-09-12T02:19:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.075+0000 7f0225e55740 1 Processor -- start 2024-09-12T02:19:25.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.075+0000 7f0225e55740 1 -- start start 2024-09-12T02:19:25.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee51b40 0x5614eee50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee505d0 0x5614eee50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5614eedb62f0 con 0x5614eee5b370 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614eed5dcf0 con 0x5614eee51b40 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0225e55740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5614eedb22b0 con 0x5614eee505d0 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37060/0 (socket says 172.21.15.3:37060) 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 -- 172.21.15.3:0/3826053095 learned_addr learned my addr 172.21.15.3:0/3826053095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0212ffd640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 0x7f02080a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 -- 172.21.15.3:0/3826053095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee505d0 msgr2=0x5614eee50a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee505d0 0x5614eee50a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 -- 172.21.15.3:0/3826053095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee51b40 msgr2=0x5614eee50090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee51b40 0x5614eee50090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.076+0000 7f0221017640 1 -- 172.21.15.3:0/3826053095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614eed423a0 con 0x5614eee5b370 2024-09-12T02:19:25.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0212ffd640 1 -- 172.21.15.3:0/3449649994 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f021801faf0 con 0x7f021c100430 2024-09-12T02:19:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0221017640 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b690 tx=0x7f020c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64165e5160bc6552 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0207fff640 1 -- 172.21.15.3:0/3826053095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c003bd0 con 0x5614eee5b370 2024-09-12T02:19:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0222019640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 0x7f02080a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0207fff640 1 -- 172.21.15.3:0/3826053095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c015070 con 0x5614eee5b370 2024-09-12T02:19:25.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.077+0000 7f0207fff640 1 -- 172.21.15.3:0/3826053095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c011440 con 0x5614eee5b370 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 -- 172.21.15.3:0/3826053095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 msgr2=0x5614eee5d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0222019640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 0x7f02080a52d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0214004af0 tx=0x7f0214004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b690 tx=0x7f020c00bb50 comp rx=0 tx=0).stop 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 -- 172.21.15.3:0/3826053095 shutdown_connections 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee5d760 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee505d0 0x5614eee50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 --2- 172.21.15.3:0/3826053095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee51b40 0x5614eee50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.078+0000 7f0225e55740 1 -- 172.21.15.3:0/3826053095 >> 172.21.15.3:0/3826053095 conn(0x5614eee19310 msgr2=0x5614eee4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.079+0000 7f0225e55740 1 -- 172.21.15.3:0/3826053095 shutdown_connections 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.079+0000 7f0225e55740 1 -- 172.21.15.3:0/3826053095 wait complete. 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.079+0000 7f0225e55740 1 -- start start 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.079+0000 7f0225e55740 1 Processor -- start 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee5add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee58810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5614eee5e070 con 0x5614eee5b370 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5614eed71060 con 0x5614eee505d0 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5614eee5e1f0 con 0x5614eee51b40 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0225e55740 10 client.0 _ll_register_callbacks cb 0x5614eeac48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:25.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0221017640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0221017640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:52418/0 (socket says 172.21.15.3:52418) 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0221017640 1 -- 172.21.15.3:0/2681561203 learned_addr learned my addr 172.21.15.3:0/2681561203 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0221818640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee5add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.080+0000 7f0222019640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee58810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 -- 172.21.15.3:0/2681561203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 msgr2=0x5614eee5add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee5add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 -- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 msgr2=0x5614eee58810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee58810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 -- 172.21.15.3:0/2681561203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614eeb273c0 con 0x5614eee51b40 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0222019640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee58810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0221017640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f020c004ac0 tx=0x7f020c003bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bac0cfc2a6164f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f02057fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c028060 con 0x5614eee51b40 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.081+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020c015070 con 0x5614eee51b40 2024-09-12T02:19:25.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.082+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f020c004e40 con 0x5614eee51b40 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.082+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f020c005040 con 0x5614eee51b40 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.082+0000 7f0225e55740 10 client.25360 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.082+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614eee69b90 con 0x5614eee51b40 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.082+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c015070 con 0x5614eee51b40 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f020c022070 con 0x5614eee51b40 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 10 client.25360 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 10 client.25360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 20 client.25360 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eee6aee0 0x5614eee6d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 4 client.25360 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614eedc3860 con 0x5614eee6aee0 2024-09-12T02:19:25.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.083+0000 7f0221818640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eee6aee0 0x5614eee6d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.084+0000 7f02057fa640 10 client.25360 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.084+0000 7f0221818640 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eee6aee0 0x5614eee6d2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3736dfd78a68b6ba server_cookie=afcac14a11877ee6 in_seq=0 out_seq=0 2024-09-12T02:19:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f02057fa640 1 -- 172.21.15.3:0/2681561203 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614eedc3860 con 0x5614eee6aee0 2024-09-12T02:19:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f02057fa640 10 client.25360 handle_command_reply: tid=0 2024-09-12T02:19:25.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 client.25360 shutdown 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 msgr2=0x5614eee58080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f020c004ac0 tx=0x7f020c003bd0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 shutdown_connections 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee58810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eee6aee0 0x5614eee6d2d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee58080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 --2- 172.21.15.3:0/2681561203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee5add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 >> 172.21.15.3:0/2681561203 conn(0x5614eee19310 msgr2=0x5614eee52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f02057fa640 0 client.25360 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.085+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 shutdown_connections 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.086+0000 7f0225e55740 1 -- 172.21.15.3:0/2681561203 wait complete. 2024-09-12T02:19:25.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.086+0000 7f0225e55740 20 client.25360 trim_cache size 0 max 16384 2024-09-12T02:19:25.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.112+0000 7f0225e55740 1 Processor -- start 2024-09-12T02:19:25.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.112+0000 7f0225e55740 1 -- start start 2024-09-12T02:19:25.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.112+0000 7f0225e55740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.112+0000 7f0225e55740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee4bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0225e55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0225e55740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ab410 con 0x5614eee5b370 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0225e55740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1acf50 con 0x5614eee505d0 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0225e55740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1abae0 con 0x5614eee51b40 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee4fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35300/0 (socket says 172.21.15.3:35300) 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221818640 1 -- 172.21.15.3:0/791397175 learned_addr learned my addr 172.21.15.3:0/791397175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee4bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0222019640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 -- 172.21.15.3:0/791397175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 msgr2=0x5614eee4fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee4fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 -- 172.21.15.3:0/791397175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 msgr2=0x5614eee6ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee6ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0221017640 1 -- 172.21.15.3:0/791397175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614eee4c510 con 0x5614eee51b40 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.113+0000 7f0222019640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.114+0000 7f0221017640 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee4bbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b690 tx=0x7f020c00bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f7912e5ede3e4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.114+0000 7f02077fe640 1 -- 172.21.15.3:0/791397175 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c028040 con 0x5614eee51b40 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.114+0000 7f02077fe640 1 -- 172.21.15.3:0/791397175 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c015070 con 0x5614eee51b40 2024-09-12T02:19:25.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.114+0000 7f02077fe640 1 -- 172.21.15.3:0/791397175 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c0116a0 con 0x5614eee51b40 2024-09-12T02:19:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 -- 172.21.15.3:0/791397175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 msgr2=0x5614eee4bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee4bbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b690 tx=0x7f020c00bb50 comp rx=0 tx=0).stop 2024-09-12T02:19:25.121 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 -- 172.21.15.3:0/791397175 shutdown_connections 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee5b370 0x5614eee6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee51b40 0x5614eee4bbf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 --2- 172.21.15.3:0/791397175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 -- 172.21.15.3:0/791397175 >> 172.21.15.3:0/791397175 conn(0x5614eee19310 msgr2=0x5614eee4b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 -- 172.21.15.3:0/791397175 shutdown_connections 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.115+0000 7f0225e55740 1 -- 172.21.15.3:0/791397175 wait complete. 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.116+0000 7f0225e55740 1 -- start start 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.116+0000 7f0225e55740 1 Processor -- start 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0221017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0221818640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35312/0 (socket says 172.21.15.3:35312) 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0221017640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37092/0 (socket says 172.21.15.3:37092) 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0221818640 1 -- 172.21.15.3:0/1860756288 learned_addr learned my addr 172.21.15.3:0/1860756288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee5b370 0x5614eee6d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ac880 con 0x5614eee51b40 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ac1b0 con 0x5614eee505d0 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1a93f0 con 0x5614eee5b370 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.117+0000 7f0222019640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee5b370 0x5614eee6d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 -- 172.21.15.3:0/1860756288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee5b370 msgr2=0x5614eee6d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee5b370 0x5614eee6d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 -- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 msgr2=0x5614eee70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0225e55740 10 client.0 _ll_register_callbacks cb 0x7f020c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:25.123 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614eee6e0f0 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221017640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020c015070 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0221818640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f02180d4c40 tx=0x7f0218000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3247baa2bd3e12fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f02057fa640 10 client.25283 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f02057fa640 1 -- 172.21.15.3:0/1860756288 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02180a7cc0 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f02057fa640 1 -- 172.21.15.3:0/1860756288 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f02180c22b0 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f02057fa640 1 -- 172.21.15.3:0/1860756288 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02180ac580 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0225e55740 10 client.25283 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.118+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5614eeed5330 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f02057fa640 1 -- 172.21.15.3:0/1860756288 <== mon.2 v2:172.21.15.26:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f021809ff80 con 0x5614eee505d0 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f0225e55740 10 client.25283 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:25.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f0225e55740 10 client.25283 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f0225e55740 20 client.25283 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eeed5990 0x5614eeed7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.119+0000 7f0221017640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eeed5990 0x5614eeed7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.120+0000 7f0225e55740 4 client.25283 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.120+0000 7f02057fa640 10 client.25283 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.120+0000 7f0221017640 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eeed5990 0x5614eeed7e50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee66b5ee78cd691e server_cookie=abae9f23fbec29c in_seq=0 out_seq=0 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.120+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614eedbdd70 con 0x5614eeed5990 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.120+0000 7f02057fa640 1 -- 172.21.15.3:0/1860756288 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614eedbdd70 con 0x5614eeed5990 2024-09-12T02:19:25.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f02057fa640 10 client.25283 handle_command_reply: tid=0 2024-09-12T02:19:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 client.25283 shutdown 2024-09-12T02:19:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 msgr2=0x5614eee702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f02180d4c40 tx=0x7f0218000bc0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 shutdown_connections 2024-09-12T02:19:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x5614eeed5990 0x5614eeed7e50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f02057fa640 0 client.25283 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5614eee5b370 0x5614eee6d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5614eee51b40 0x5614eee70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.127 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 --2- 172.21.15.3:0/1860756288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5614eee505d0 0x5614eee702d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 >> 172.21.15.3:0/1860756288 conn(0x5614eee19310 msgr2=0x5614eee4bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 shutdown_connections 2024-09-12T02:19:25.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 1 -- 172.21.15.3:0/1860756288 wait complete. 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.121+0000 7f0225e55740 20 client.25283 trim_cache size 0 max 16384 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 msgr2=0x7f02080a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 0x7f02080a52d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0214004af0 tx=0x7f0214004a80 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 msgr2=0x7f021c10e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f021800bf20 tx=0x7f021800dd60 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 shutdown_connections 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f02080a2e10 0x7f02080a52d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f021c105560 0x7f021c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f021c100de0 0x7f021c10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 --2- 172.21.15.3:0/3449649994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f021c100430 0x7f021c10e800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.122+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 >> 172.21.15.3:0/3449649994 conn(0x7f021c077230 msgr2=0x7f021c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.123+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 shutdown_connections 2024-09-12T02:19:25.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:25.123+0000 7f0223a7c640 1 -- 172.21.15.3:0/3449649994 wait complete. 2024-09-12T02:19:25.883 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:19:26.258 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:26.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:26 smithi026 ceph-mon[35496]: pgmap v367: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:26 smithi003 ceph-mon[31476]: pgmap v367: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:26 smithi092 ceph-mon[36089]: pgmap v367: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:26.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 -- 172.21.15.3:0/330270539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 msgr2=0x7fb02410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 --2- 172.21.15.3:0/330270539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02410e620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009ca0 tx=0x7fb01c02a950 comp rx=0 tx=0).stop 2024-09-12T02:19:26.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 -- 172.21.15.3:0/330270539 shutdown_connections 2024-09-12T02:19:26.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 --2- 172.21.15.3:0/330270539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02410e620 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 --2- 172.21.15.3:0/330270539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024103670 0x7fb02410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 --2- 172.21.15.3:0/330270539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024102d50 0x7fb024103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.796+0000 7fb029ab7640 1 -- 172.21.15.3:0/330270539 >> 172.21.15.3:0/330270539 conn(0x7fb0240fc7d0 msgr2=0x7fb0240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.797+0000 7fb029ab7640 1 -- 172.21.15.3:0/330270539 shutdown_connections 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.797+0000 7fb029ab7640 1 -- 172.21.15.3:0/330270539 wait complete. 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 Processor -- start 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 -- start start 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024102d50 0x7fb02419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024103670 0x7fb02419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb024111f60 con 0x7fb02410c180 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb024111de0 con 0x7fb024103670 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb029ab7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241120e0 con 0x7fb024102d50 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb023fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb023fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37120/0 (socket says 172.21.15.3:37120) 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb023fff640 1 -- 172.21.15.3:0/3219057287 learned_addr learned my addr 172.21.15.3:0/3219057287 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb0237fe640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024102d50 0x7fb02419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.798+0000 7fb022ffd640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024103670 0x7fb02419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 -- 172.21.15.3:0/3219057287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024102d50 msgr2=0x7fb02419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024102d50 0x7fb02419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 -- 172.21.15.3:0/3219057287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024103670 msgr2=0x7fb02419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024103670 0x7fb02419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 -- 172.21.15.3:0/3219057287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02419f5e0 con 0x7fb02410c180 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb023fff640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb01c004af0 tx=0x7fb01c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb020ff9640 1 -- 172.21.15.3:0/3219057287 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c034040 con 0x7fb02410c180 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02419f870 con 0x7fb02410c180 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb020ff9640 1 -- 172.21.15.3:0/3219057287 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb01c030920 con 0x7fb02410c180 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb020ff9640 1 -- 172.21.15.3:0/3219057287 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0047b0 con 0x7fb02410c180 2024-09-12T02:19:26.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.799+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241a71b0 con 0x7fb02410c180 2024-09-12T02:19:26.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb02be90740 1 Processor -- start 2024-09-12T02:19:26.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb020ff9640 1 -- 172.21.15.3:0/3219057287 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb01c037020 con 0x7fb02410c180 2024-09-12T02:19:26.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb02be90740 1 -- start start 2024-09-12T02:19:26.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb02be90740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73e90090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb02be90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.803+0000 7fb02be90740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e9b370 0x55ee73e9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb02be90740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ee73df62f0 con 0x55ee73e9b370 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb02be90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee73d9dcf0 con 0x55ee73e905d0 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb02be90740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ee73df22b0 con 0x55ee73e91b40 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73e90090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35328/0 (socket says 172.21.15.3:35328) 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 -- 172.21.15.3:0/2577008037 learned_addr learned my addr 172.21.15.3:0/2577008037 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb022ffd640 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e9b370 0x55ee73e9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 -- 172.21.15.3:0/2577008037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 msgr2=0x55ee73e90090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73e90090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 -- 172.21.15.3:0/2577008037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e9b370 msgr2=0x55ee73e9d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e9b370 0x55ee73e9d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.804+0000 7fb0237fe640 1 -- 172.21.15.3:0/2577008037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee73d823a0 con 0x55ee73e905d0 2024-09-12T02:19:26.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb0237fe640 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb01400bcd0 tx=0x7fb0140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5379df5d7f5942 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb009ffb640 1 -- 172.21.15.3:0/2577008037 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014005020 con 0x55ee73e905d0 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb020ff9640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 0x7fb00c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb022ffd640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 0x7fb00c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb020ff9640 1 -- 172.21.15.3:0/3219057287 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb01c0efb00 con 0x7fb02410c180 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb009ffb640 1 -- 172.21.15.3:0/2577008037 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0140047a0 con 0x55ee73e905d0 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.805+0000 7fb009ffb640 1 -- 172.21.15.3:0/2577008037 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140117a0 con 0x55ee73e905d0 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 -- 172.21.15.3:0/2577008037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 msgr2=0x55ee73e90a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb022ffd640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 0x7fb00c0a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ee73e91070 tx=0x7fb018004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb01400bcd0 tx=0x7fb0140107a0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 -- 172.21.15.3:0/2577008037 shutdown_connections 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e9b370 0x55ee73e9d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e90a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 --2- 172.21.15.3:0/2577008037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73e90090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 -- 172.21.15.3:0/2577008037 >> 172.21.15.3:0/2577008037 conn(0x55ee73e59310 msgr2=0x55ee73e8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.806+0000 7fb02be90740 1 -- 172.21.15.3:0/2577008037 shutdown_connections 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.807+0000 7fb02be90740 1 -- 172.21.15.3:0/2577008037 wait complete. 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.807+0000 7fb02be90740 1 -- start start 2024-09-12T02:19:26.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 Processor -- start 2024-09-12T02:19:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 0x55ee73e98600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ee73e9e070 con 0x55ee73e91b40 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ee73db1060 con 0x55ee73e905d0 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ee73e9e1f0 con 0x55ee73e9b370 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb02be90740 10 client.0 _ll_register_callbacks cb 0x55ee73b048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37128/0 (socket says 172.21.15.3:37128) 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb022ffd640 1 -- 172.21.15.3:0/1257060615 learned_addr learned my addr 172.21.15.3:0/1257060615 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.808+0000 7fb0237fe640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb023fff640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 0x55ee73e98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.815 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 -- 172.21.15.3:0/1257060615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 msgr2=0x55ee73e98600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 0x55ee73e98600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 -- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 msgr2=0x55ee73e9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e9ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 -- 172.21.15.3:0/1257060615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee73b673c0 con 0x55ee73e91b40 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb023fff640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 0x55ee73e98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb022ffd640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb0180042d0 tx=0x7fb0180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d5ad86119065de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018019040 con 0x55ee73e91b40 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb018005220 con 0x55ee73e91b40 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.809+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0140047a0 con 0x55ee73e91b40 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb018005420 con 0x55ee73e91b40 2024-09-12T02:19:26.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee73ea9b10 con 0x55ee73e91b40 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0140047a0 con 0x55ee73e91b40 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb018010760 con 0x55ee73e91b40 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 10 client.15735 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 20 client.15735 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73eab070 0x55ee73ead490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 4 client.15735 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.810+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee73e03860 con 0x55ee73eab070 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.811+0000 7fb0237fe640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73eab070 0x55ee73ead490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.811+0000 7fb002ffd640 10 client.15735 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.811+0000 7fb0237fe640 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73eab070 0x55ee73ead490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25348f1aa66c969f server_cookie=94c68df49c0b8a6 in_seq=0 out_seq=0 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb002ffd640 1 -- 172.21.15.3:0/1257060615 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee73e03860 con 0x55ee73eab070 2024-09-12T02:19:26.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb002ffd640 10 client.15735 handle_command_reply: tid=0 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 client.15735 shutdown 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 msgr2=0x55ee73e98010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb0180042d0 tx=0x7fb0180046c0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 shutdown_connections 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e9b370 0x55ee73e98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e91b40 0x55ee73e98010 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73eab070 0x55ee73ead490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 --2- 172.21.15.3:0/1257060615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e905d0 0x55ee73e9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb002ffd640 0 client.15735 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.812+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 >> 172.21.15.3:0/1257060615 conn(0x55ee73e59310 msgr2=0x55ee73e92960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.813+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 shutdown_connections 2024-09-12T02:19:26.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.813+0000 7fb02be90740 1 -- 172.21.15.3:0/1257060615 wait complete. 2024-09-12T02:19:26.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.813+0000 7fb02be90740 20 client.15735 trim_cache size 0 max 16384 2024-09-12T02:19:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 Processor -- start 2024-09-12T02:19:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 -- start start 2024-09-12T02:19:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ab3f0 con 0x55ee73e905d0 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241acf30 con 0x55ee73e9b370 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb02be90740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241abac0 con 0x55ee73e91b40 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37140/0 (socket says 172.21.15.3:37140) 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 -- 172.21.15.3:0/3230998619 learned_addr learned my addr 172.21.15.3:0/3230998619 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb022ffd640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb023fff640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 -- 172.21.15.3:0/3230998619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 msgr2=0x55ee73ea7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 -- 172.21.15.3:0/3230998619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 msgr2=0x55ee73e978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.839+0000 7fb0237fe640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb0237fe640 1 -- 172.21.15.3:0/3230998619 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee73e9a980 con 0x55ee73e905d0 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb023fff640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb0237fe640 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b6a0 tx=0x7fb01400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf002e5e4dded63 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb0097fa640 1 -- 172.21.15.3:0/3230998619 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01402c020 con 0x55ee73e905d0 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb0097fa640 1 -- 172.21.15.3:0/3230998619 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014005510 con 0x55ee73e905d0 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.840+0000 7fb0097fa640 1 -- 172.21.15.3:0/3230998619 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014004c40 con 0x55ee73e905d0 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- 172.21.15.3:0/3230998619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 msgr2=0x55ee73e8ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b6a0 tx=0x7fb01400b630 comp rx=0 tx=0).stop 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- 172.21.15.3:0/3230998619 shutdown_connections 2024-09-12T02:19:26.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 --2- 172.21.15.3:0/3230998619 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73e905d0 0x55ee73e8ce10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- 172.21.15.3:0/3230998619 >> 172.21.15.3:0/3230998619 conn(0x55ee73e59310 msgr2=0x55ee73e8c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- 172.21.15.3:0/3230998619 shutdown_connections 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- 172.21.15.3:0/3230998619 wait complete. 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.841+0000 7fb02be90740 1 -- start start 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 Processor -- start 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ac860 con 0x55ee73eab070 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ac190 con 0x55ee73e9b370 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241a93f0 con 0x55ee73e91b40 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb02be90740 10 client.0 _ll_register_callbacks cb 0x7fb018025f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb023fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb022ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb023fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37144/0 (socket says 172.21.15.3:37144) 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb023fff640 1 -- 172.21.15.3:0/4268275095 learned_addr learned my addr 172.21.15.3:0/4268275095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.842+0000 7fb023fff640 1 -- 172.21.15.3:0/4268275095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 msgr2=0x55ee73ea3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb0237fe640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb023fff640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb023fff640 1 -- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 msgr2=0x55ee73e94da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb023fff640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e94da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb023fff640 1 -- 172.21.15.3:0/4268275095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee73e95be0 con 0x55ee73eab070 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb022ffd640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e94da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb0237fe640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb023fff640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b8760 tx=0x7fb01c0bc1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1048167418542be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:26.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb002ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c02c040 con 0x55ee73eab070 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb01c0b8970 con 0x55ee73eab070 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb014005510 con 0x55ee73eab070 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb01c0bca00 con 0x55ee73eab070 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb02be90740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee73f15dd0 con 0x55ee73eab070 2024-09-12T02:19:26.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.843+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb014005510 con 0x55ee73eab070 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb01c0c4ae0 con 0x55ee73eab070 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 10 client.15741 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 20 client.15741 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73f164a0 0x55ee73f18890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 4 client.15741 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee73dfdd70 con 0x55ee73f164a0 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb0237fe640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73f164a0 0x55ee73f18890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb002ffd640 10 client.15741 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.844+0000 7fb0237fe640 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73f164a0 0x55ee73f18890 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9b6461a9493328e server_cookie=cd80ba44468a06f0 in_seq=0 out_seq=0 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb002ffd640 1 -- 172.21.15.3:0/4268275095 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee73dfdd70 con 0x55ee73f164a0 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb002ffd640 10 client.15741 handle_command_reply: tid=0 2024-09-12T02:19:26.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 client.15741 shutdown 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 msgr2=0x55ee73e953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0b8760 tx=0x7fb01c0bc1c0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 shutdown_connections 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ee73eab070 0x55ee73e953b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55ee73f164a0 0x55ee73f18890 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ee73e9b370 0x55ee73e94da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 --2- 172.21.15.3:0/4268275095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ee73e91b40 0x55ee73ea3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 >> 172.21.15.3:0/4268275095 conn(0x55ee73e59310 msgr2=0x55ee73e99870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb002ffd640 0 client.15741 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:26.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 shutdown_connections 2024-09-12T02:19:26.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 1 -- 172.21.15.3:0/4268275095 wait complete. 2024-09-12T02:19:26.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.845+0000 7fb02be90740 20 client.15741 trim_cache size 0 max 16384 2024-09-12T02:19:26.854 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:26.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 msgr2=0x7fb00c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 0x7fb00c0a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ee73e91070 tx=0x7fb018004730 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 msgr2=0x7fb02419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb01c004af0 tx=0x7fb01c02f5b0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 shutdown_connections 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb00c0a2c90 0x7fb00c0a5150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb02410c180 0x7fb02419ef00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb024103670 0x7fb02419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 --2- 172.21.15.3:0/3219057287 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb024102d50 0x7fb02419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 >> 172.21.15.3:0/3219057287 conn(0x7fb0240fc7d0 msgr2=0x7fb0241008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 shutdown_connections 2024-09-12T02:19:26.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:26.846+0000 7fb029ab7640 1 -- 172.21.15.3:0/3219057287 wait complete. 2024-09-12T02:19:27.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:19:27.890 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:28.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:28 smithi003 ceph-mon[31476]: pgmap v368: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-12T02:19:28.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.426+0000 7f4d71d51640 1 -- 172.21.15.3:0/3684526073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 msgr2=0x7f4d6c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.426+0000 7f4d71d51640 1 --2- 172.21.15.3:0/3684526073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c1067d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4d60009900 tx=0x7f4d6002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 -- 172.21.15.3:0/3684526073 shutdown_connections 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 --2- 172.21.15.3:0/3684526073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c106f00 0x7f4d6c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 --2- 172.21.15.3:0/3684526073 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c1067d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 --2- 172.21.15.3:0/3684526073 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c101dd0 0x7f4d6c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 -- 172.21.15.3:0/3684526073 >> 172.21.15.3:0/3684526073 conn(0x7f4d6c0fd8b0 msgr2=0x7f4d6c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 -- 172.21.15.3:0/3684526073 shutdown_connections 2024-09-12T02:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.427+0000 7f4d71d51640 1 -- 172.21.15.3:0/3684526073 wait complete. 2024-09-12T02:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d71d51640 1 Processor -- start 2024-09-12T02:19:28.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d71d51640 1 -- start start 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d71d51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c101dd0 0x7f4d6c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d71d51640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d71d51640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 0x7f4d6c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.428+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37162/0 (socket says 172.21.15.3:37162) 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 -- 172.21.15.3:0/983336316 learned_addr learned my addr 172.21.15.3:0/983336316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c111f60 con 0x7f4d6c102780 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c111de0 con 0x7f4d6c101dd0 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1120e0 con 0x7f4d6c106f00 2024-09-12T02:19:28.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c101dd0 0x7f4d6c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 0x7f4d6c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 -- 172.21.15.3:0/983336316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 msgr2=0x7f4d6c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 0x7f4d6c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 -- 172.21.15.3:0/983336316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c101dd0 msgr2=0x7f4d6c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c101dd0 0x7f4d6c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c19d3f0 con 0x7f4d6c102780 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 0x7f4d6c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.429+0000 7f4d6affd640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d60004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.430+0000 7f4d68ff9640 1 -- 172.21.15.3:0/983336316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6003f030 con 0x7f4d6c102780 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.430+0000 7f4d68ff9640 1 -- 172.21.15.3:0/983336316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d60004440 con 0x7f4d6c102780 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.430+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c19d680 con 0x7f4d6c102780 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.430+0000 7f4d68ff9640 1 -- 172.21.15.3:0/983336316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60035070 con 0x7f4d6c102780 2024-09-12T02:19:28.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.431+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c19daf0 con 0x7f4d6c102780 2024-09-12T02:19:28.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.433+0000 7f4d68ff9640 1 -- 172.21.15.3:0/983336316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f4d60042030 con 0x7f4d6c102780 2024-09-12T02:19:28.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.434+0000 7f4d68ff9640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 0x7f4d540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.434+0000 7f4d7412a740 1 Processor -- start 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 0x7f4d540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 -- start start 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6cf7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d0cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dce6c5d2f0 con 0x55dce6d0ab80 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d7412a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dce6c04cf0 con 0x55dce6cf8b40 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d0cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d0cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37168/0 (socket says 172.21.15.3:37168) 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.435+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:52516/0 (socket says 172.21.15.3:52516) 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6bfff640 1 -- 172.21.15.3:0/2673826621 learned_addr learned my addr 172.21.15.3:0/2673826621 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dce6c592b0 con 0x55dce6cf75d0 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 0x7f4d540a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009bd0 tx=0x7f4d5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6cf7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d68ff9640 1 -- 172.21.15.3:0/983336316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4d600ef550 con 0x7f4d6c102780 2024-09-12T02:19:28.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 -- 172.21.15.3:0/2673826621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 msgr2=0x55dce6cf7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6cf7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 -- 172.21.15.3:0/2673826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 msgr2=0x55dce6d0cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d0cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 -- 172.21.15.3:0/2673826621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce6be93a0 con 0x55dce6cf75d0 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.436+0000 7f4d6affd640 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d600ecef0 tx=0x7f4d6003b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4f8c9217f01831 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.437+0000 7f4d51ffb640 1 -- 172.21.15.3:0/2673826621 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60049040 con 0x55dce6cf75d0 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.437+0000 7f4d51ffb640 1 -- 172.21.15.3:0/2673826621 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d600bc340 con 0x55dce6cf75d0 2024-09-12T02:19:28.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.437+0000 7f4d51ffb640 1 -- 172.21.15.3:0/2673826621 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003cd0 con 0x55dce6cf75d0 2024-09-12T02:19:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.438+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 msgr2=0x55dce6cf7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.438+0000 7f4d7412a740 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d600ecef0 tx=0x7f4d6003b4d0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 shutdown_connections 2024-09-12T02:19:28.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d0cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf75d0 0x55dce6cf7a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 --2- 172.21.15.3:0/2673826621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6cf7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 >> 172.21.15.3:0/2673826621 conn(0x55dce6cc0310 msgr2=0x55dce6cf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 shutdown_connections 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 -- 172.21.15.3:0/2673826621 wait complete. 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 -- start start 2024-09-12T02:19:28.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.439+0000 7f4d7412a740 1 Processor -- start 2024-09-12T02:19:28.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6d0ab80 0x55dce6d08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dce6d0d640 con 0x55dce6cf75d0 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55dce6c18060 con 0x55dce6d0ab80 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55dce6d0d7c0 con 0x55dce6cf8b40 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d7412a740 10 client.0 _ll_register_callbacks cb 0x55dce696b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:52520/0 (socket says 172.21.15.3:52520) 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 -- 172.21.15.3:0/3026919919 learned_addr learned my addr 172.21.15.3:0/3026919919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6d0ab80 0x55dce6d08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 -- 172.21.15.3:0/3026919919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6d0ab80 msgr2=0x55dce6d08050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6d0ab80 0x55dce6d08050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 -- 172.21.15.3:0/3026919919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 msgr2=0x55dce6d0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d0a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.440+0000 7f4d6affd640 1 -- 172.21.15.3:0/3026919919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce69ce3c0 con 0x55dce6cf8b40 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d6affd640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d600c4380 tx=0x7f4d600ed370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1924f1992c4fbe51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d60049070 con 0x55dce6cf8b40 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4d600a6c80 con 0x55dce6cf8b40 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d600bc340 con 0x55dce6cf8b40 2024-09-12T02:19:28.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d600bbc40 con 0x55dce6cf8b40 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d7412a740 10 client.25390 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dce6d0d640 con 0x55dce6cf8b40 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.441+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d600c48b0 con 0x55dce6cf8b40 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d600c4490 con 0x55dce6cf8b40 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 10 client.25390 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 10 client.25390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 20 client.25390 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 4 client.25390 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4d600bc340 con 0x55dce6d1a2c0 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.442+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d1c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.443+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d1c6d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e57205262b9bac79 server_cookie=74858f2906df1c9e in_seq=0 out_seq=0 2024-09-12T02:19:28.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.443+0000 7f4d4affd640 10 client.25390 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d4affd640 1 -- 172.21.15.3:0/3026919919 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4d600bc340 con 0x55dce6d1a2c0 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d4affd640 10 client.25390 handle_command_reply: tid=0 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 client.25390 shutdown 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 msgr2=0x55dce6d078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d600c4380 tx=0x7f4d600ed370 comp rx=0 tx=0).stop 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 shutdown_connections 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6d0ab80 0x55dce6d08050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d1c6d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d4affd640 0 client.25390 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d078c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3026919919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 >> 172.21.15.3:0/3026919919 conn(0x55dce6cc0310 msgr2=0x55dce6cf9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 shutdown_connections 2024-09-12T02:19:28.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 1 -- 172.21.15.3:0/3026919919 wait complete. 2024-09-12T02:19:28.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.444+0000 7f4d7412a740 20 client.25390 trim_cache size 0 max 16384 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 Processor -- start 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 -- start start 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf75d0 0x55dce6d068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d21fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ab390 con 0x55dce6d0ab80 2024-09-12T02:19:28.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1aced0 con 0x55dce6cf75d0 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d7412a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1aba60 con 0x55dce6cf8b40 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45474/0 (socket says 172.21.15.3:45474) 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d6bfff640 1 -- 172.21.15.3:0/3360540732 learned_addr learned my addr 172.21.15.3:0/3360540732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.468+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf75d0 0x55dce6d068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6affd640 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d21fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 -- 172.21.15.3:0/3360540732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 msgr2=0x55dce6d21fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d21fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 -- 172.21.15.3:0/3360540732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf75d0 msgr2=0x55dce6d068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf75d0 0x55dce6d068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 -- 172.21.15.3:0/3360540732 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce6cf4960 con 0x55dce6d0ab80 2024-09-12T02:19:28.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d6bfff640 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d64012450 tx=0x7f4d64012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0dab9410c4286f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d517fa640 1 -- 172.21.15.3:0/3360540732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64020070 con 0x55dce6d0ab80 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d517fa640 1 -- 172.21.15.3:0/3360540732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d64010a30 con 0x55dce6d0ab80 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.469+0000 7f4d517fa640 1 -- 172.21.15.3:0/3360540732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6401bac0 con 0x55dce6d0ab80 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 -- 172.21.15.3:0/3360540732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 msgr2=0x55dce6d22500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4d64012450 tx=0x7f4d64012910 comp rx=0 tx=0).stop 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 -- 172.21.15.3:0/3360540732 shutdown_connections 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6d0ab80 0x55dce6d22500 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6cf8b40 0x55dce6d21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 --2- 172.21.15.3:0/3360540732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf75d0 0x55dce6d068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.470+0000 7f4d7412a740 1 -- 172.21.15.3:0/3360540732 >> 172.21.15.3:0/3360540732 conn(0x55dce6cc0310 msgr2=0x55dce6d06190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.471+0000 7f4d7412a740 1 -- 172.21.15.3:0/3360540732 shutdown_connections 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.471+0000 7f4d7412a740 1 -- 172.21.15.3:0/3360540732 wait complete. 2024-09-12T02:19:28.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.471+0000 7f4d7412a740 1 -- start start 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.471+0000 7f4d7412a740 1 Processor -- start 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6d04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 0x55dce6d05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ac800 con 0x55dce6cf75d0 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1ac130 con 0x55dce6cf8b40 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1a9370 con 0x55dce6d0ab80 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d7412a740 10 client.0 _ll_register_callbacks cb 0x7f4d64009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 0x55dce6d05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 0x55dce6d05310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:54746/0 (socket says 172.21.15.3:54746) 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d6bfff640 1 -- 172.21.15.3:0/88438028 learned_addr learned my addr 172.21.15.3:0/88438028 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.472+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6affd640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6d04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 -- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 msgr2=0x55dce6d05310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 0x55dce6d05310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 -- 172.21.15.3:0/88438028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 msgr2=0x55dce6d04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:28.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6d04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 -- 172.21.15.3:0/88438028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce6d83f90 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6affd640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6d04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d6b7fe640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d04890 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b9ef0 tx=0x7f4d5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc000ce97ea299bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.473+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c02a570 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4d5c009040 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d64010a30 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c02a710 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d7412a740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dce6d85250 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d64010a30 con 0x55dce6cf75d0 2024-09-12T02:19:28.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.474+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5c03a050 con 0x55dce6cf75d0 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 20 client.15759 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 4 client.15759 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:28.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dce6c6a860 con 0x55dce6d1a2c0 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d6affd640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d879b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d4affd640 10 client.15759 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.475+0000 7f4d6affd640 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d879b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff49190532f2e416 server_cookie=c18b75864f76e1c0 in_seq=0 out_seq=0 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.476+0000 7f4d4affd640 1 -- 172.21.15.3:0/88438028 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dce6c6a860 con 0x55dce6d1a2c0 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.476+0000 7f4d4affd640 10 client.15759 handle_command_reply: tid=0 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.476+0000 7f4d7412a740 1 client.15759 shutdown 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.476+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 msgr2=0x55dce6d04890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.476+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d04890 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4d600b9ef0 tx=0x7f4d5c004730 comp rx=0 tx=0).stop 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 shutdown_connections 2024-09-12T02:19:28.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x55dce6d1a2c0 0x55dce6d879b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d4affd640 0 client.15759 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55dce6d0ab80 0x55dce6d05310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dce6cf8b40 0x55dce6d04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 --2- 172.21.15.3:0/88438028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dce6cf75d0 0x55dce6d04890 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 >> 172.21.15.3:0/88438028 conn(0x55dce6cc0310 msgr2=0x55dce6d09120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 shutdown_connections 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 1 -- 172.21.15.3:0/88438028 wait complete. 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.477+0000 7f4d7412a740 20 client.15759 trim_cache size 0 max 16384 2024-09-12T02:19:28.485 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 msgr2=0x7f4d540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 0x7f4d540a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009bd0 tx=0x7f4d5c005f90 comp rx=0 tx=0).stop 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 msgr2=0x7f4d6c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d60006010 tx=0x7f4d60004040 comp rx=0 tx=0).stop 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 shutdown_connections 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4d540a2c90 0x7f4d540a5150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6c106f00 0x7f4d6c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4d6c102780 0x7f4d6c198980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 --2- 172.21.15.3:0/983336316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d6c101dd0 0x7f4d6c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.478+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 >> 172.21.15.3:0/983336316 conn(0x7f4d6c0fd8b0 msgr2=0x7f4d6c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.479+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 shutdown_connections 2024-09-12T02:19:28.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:28.479+0000 7f4d71d51640 1 -- 172.21.15.3:0/983336316 wait complete. 2024-09-12T02:19:28.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:28 smithi092 ceph-mon[36089]: pgmap v368: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-12T02:19:28.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:28 smithi026 ceph-mon[35496]: pgmap v368: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-12T02:19:29.197 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:19:29.594 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.077+0000 7fe712c78640 1 -- 172.21.15.3:0/783853714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c1037a0 msgr2=0x7fe70c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.077+0000 7fe712c78640 1 --2- 172.21.15.3:0/783853714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c1037a0 0x7fe70c10bc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009900 tx=0x7fe6f802a410 comp rx=0 tx=0).stop 2024-09-12T02:19:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 -- 172.21.15.3:0/783853714 shutdown_connections 2024-09-12T02:19:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 --2- 172.21.15.3:0/783853714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 0x7fe70c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 --2- 172.21.15.3:0/783853714 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c1037a0 0x7fe70c10bc00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 --2- 172.21.15.3:0/783853714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c102df0 0x7fe70c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 -- 172.21.15.3:0/783853714 >> 172.21.15.3:0/783853714 conn(0x7fe70c0fc7d0 msgr2=0x7fe70c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 -- 172.21.15.3:0/783853714 shutdown_connections 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 -- 172.21.15.3:0/783853714 wait complete. 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 Processor -- start 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.078+0000 7fe712c78640 1 -- start start 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c1037a0 0x7fe70c100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 0x7fe70c1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1120b0 con 0x7fe70c102df0 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c111f30 con 0x7fe70c10c2f0 2024-09-12T02:19:30.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe712c78640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112230 con 0x7fe70c1037a0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45488/0 (socket says 172.21.15.3:45488) 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 -- 172.21.15.3:0/1785645913 learned_addr learned my addr 172.21.15.3:0/1785645913 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe70bfff640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c1037a0 0x7fe70c100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe711215640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 0x7fe70c1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 -- 172.21.15.3:0/1785645913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c1037a0 msgr2=0x7fe70c100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c1037a0 0x7fe70c100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 -- 172.21.15.3:0/1785645913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 msgr2=0x7fe70c1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 0x7fe70c1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.079+0000 7fe710a14640 1 -- 172.21.15.3:0/1785645913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c1a6ab0 con 0x7fe70c102df0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe710a14640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d950 tx=0x7fe70000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe709ffb640 1 -- 172.21.15.3:0/1785645913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700017030 con 0x7fe70c102df0 2024-09-12T02:19:30.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1a6da0 con 0x7fe70c102df0 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe709ffb640 1 -- 172.21.15.3:0/1785645913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe700004020 con 0x7fe70c102df0 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe709ffb640 1 -- 172.21.15.3:0/1785645913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7000047a0 con 0x7fe70c102df0 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.080+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a7200 con 0x7fe70c102df0 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.083+0000 7fe709ffb640 1 -- 172.21.15.3:0/1785645913 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fe7000041d0 con 0x7fe70c102df0 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.083+0000 7fe715051740 1 Processor -- start 2024-09-12T02:19:30.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 -- start start 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 0x555613358a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55561336bb80 0x55561336df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5556132be2f0 con 0x555613359b40 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555613265cf0 con 0x55561336bb80 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe715051740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5556132ba2b0 con 0x5556133585d0 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe711215640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe711215640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45496/0 (socket says 172.21.15.3:45496) 2024-09-12T02:19:30.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe711215640 1 -- 172.21.15.3:0/258176262 learned_addr learned my addr 172.21.15.3:0/258176262 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe709ffb640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 0x7fe6fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe709ffb640 1 -- 172.21.15.3:0/1785645913 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe7000cebc0 con 0x7fe70c102df0 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.084+0000 7fe710a14640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 0x555613358a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe711215640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 0x7fe6fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe70bfff640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55561336bb80 0x55561336df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 -- 172.21.15.3:0/258176262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55561336bb80 msgr2=0x55561336df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55561336bb80 0x55561336df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 -- 172.21.15.3:0/258176262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 msgr2=0x555613358090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 -- 172.21.15.3:0/258176262 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561324a3a0 con 0x5556133585d0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe711215640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.085+0000 7fe710a14640 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 0x555613358a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7000cc560 tx=0x7fe700004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a37bf6c7b29beb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.086+0000 7fe711215640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 0x7fe6fc0a50d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe704005c40 tx=0x7fe704004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.086+0000 7fe6f6ffd640 1 -- 172.21.15.3:0/258176262 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70009b2b0 con 0x5556133585d0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.086+0000 7fe6f6ffd640 1 -- 172.21.15.3:0/258176262 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70009f660 con 0x5556133585d0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.086+0000 7fe6f6ffd640 1 -- 172.21.15.3:0/258176262 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7000bb0b0 con 0x5556133585d0 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 -- 172.21.15.3:0/258176262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 msgr2=0x555613358a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 0x555613358a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7000cc560 tx=0x7fe700004540 comp rx=0 tx=0).stop 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 -- 172.21.15.3:0/258176262 shutdown_connections 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55561336bb80 0x55561336df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5556133585d0 0x555613358a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 --2- 172.21.15.3:0/258176262 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613358090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 -- 172.21.15.3:0/258176262 >> 172.21.15.3:0/258176262 conn(0x555613321310 msgr2=0x555613354c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 -- 172.21.15.3:0/258176262 shutdown_connections 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.087+0000 7fe715051740 1 -- 172.21.15.3:0/258176262 wait complete. 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 -- start start 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 Processor -- start 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613369000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55561336e880 con 0x555613359b40 2024-09-12T02:19:30.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555613279060 con 0x5556133585d0 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55561336ea00 con 0x55561336bb80 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.088+0000 7fe715051740 10 client.0 _ll_register_callbacks cb 0x555612fcc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45506/0 (socket says 172.21.15.3:45506) 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 -- 172.21.15.3:0/2261611256 learned_addr learned my addr 172.21.15.3:0/2261611256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe710a14640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe711215640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613369000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 -- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 msgr2=0x555613369000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613369000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 -- 172.21.15.3:0/2261611256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 msgr2=0x55561336b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 -- 172.21.15.3:0/2261611256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561302f3c0 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe710a14640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0697d0 tx=0x7fe6f8032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c32106244dc8924 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.089+0000 7fe6ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8035040 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe6f8031970 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70009f660 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6f8004040 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe715051740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55561337a370 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70009f660 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.090+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe6f8004950 con 0x555613359b40 2024-09-12T02:19:30.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 20 client.15771 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55561337b830 0x55561337dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 4 client.15771 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556132cb860 con 0x55561337b830 2024-09-12T02:19:30.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe710a14640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55561337b830 0x55561337dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe710a14640 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55561337b830 0x55561337dc40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3479bf96f83c5fc6 server_cookie=1306345fac985745 in_seq=0 out_seq=0 2024-09-12T02:19:30.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.091+0000 7fe6ebfff640 10 client.15771 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2261611256 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556132cb860 con 0x55561337b830 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe6ebfff640 10 client.15771 handle_command_reply: tid=0 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 client.15771 shutdown 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 msgr2=0x555613368870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0697d0 tx=0x7fe6f8032ed0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 shutdown_connections 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55561337b830 0x55561337dc40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613369000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613368870 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 --2- 172.21.15.3:0/2261611256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 >> 172.21.15.3:0/2261611256 conn(0x555613321310 msgr2=0x55561335a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.092+0000 7fe6ebfff640 0 client.15771 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.093+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 shutdown_connections 2024-09-12T02:19:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.093+0000 7fe715051740 1 -- 172.21.15.3:0/2261611256 wait complete. 2024-09-12T02:19:30.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.093+0000 7fe715051740 20 client.15771 trim_cache size 0 max 16384 2024-09-12T02:19:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 Processor -- start 2024-09-12T02:19:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 -- start start 2024-09-12T02:19:30.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x555613354e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555613359b40 0x555613383510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ab450 con 0x55561336bb80 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1acf90 con 0x5556133585d0 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe715051740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1abb20 con 0x555613359b40 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe711215640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe711215640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45516/0 (socket says 172.21.15.3:45516) 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe711215640 1 -- 172.21.15.3:0/2558338647 learned_addr learned my addr 172.21.15.3:0/2558338647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe710a14640 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x555613354e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.106+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555613359b40 0x555613383510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 -- 172.21.15.3:0/2558338647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555613359b40 msgr2=0x555613383510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555613359b40 0x555613383510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 -- 172.21.15.3:0/2558338647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 msgr2=0x555613354e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x555613354e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 -- 172.21.15.3:0/2558338647 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555613369370 con 0x55561336bb80 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe711215640 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe7040146e0 tx=0x7fe704009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9967e610851432 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe6f67fc640 1 -- 172.21.15.3:0/2558338647 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704011c30 con 0x55561336bb80 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe6f67fc640 1 -- 172.21.15.3:0/2558338647 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704011dd0 con 0x55561336bb80 2024-09-12T02:19:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.107+0000 7fe6f67fc640 1 -- 172.21.15.3:0/2558338647 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704021710 con 0x55561336bb80 2024-09-12T02:19:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 -- 172.21.15.3:0/2558338647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 msgr2=0x555613383a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe7040146e0 tx=0x7fe704009880 comp rx=0 tx=0).stop 2024-09-12T02:19:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 -- 172.21.15.3:0/2558338647 shutdown_connections 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55561336bb80 0x555613383a50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555613359b40 0x555613383510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 --2- 172.21.15.3:0/2558338647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x555613354e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 -- 172.21.15.3:0/2558338647 >> 172.21.15.3:0/2558338647 conn(0x555613321310 msgr2=0x555613354700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 -- 172.21.15.3:0/2558338647 shutdown_connections 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.108+0000 7fe715051740 1 -- 172.21.15.3:0/2558338647 wait complete. 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 -- start start 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 Processor -- start 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613370050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613370590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ac8c0 con 0x555613359b40 2024-09-12T02:19:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1ac1f0 con 0x5556133585d0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.109+0000 7fe715051740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1a9430 con 0x55561336bb80 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:39972/0 (socket says 172.21.15.3:39972) 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe715051740 10 client.0 _ll_register_callbacks cb 0x7fe6f80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 -- 172.21.15.3:0/2875012095 learned_addr learned my addr 172.21.15.3:0/2875012095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613370050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe711215640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613370590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 -- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 msgr2=0x555613370590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613370590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 -- 172.21.15.3:0/2875012095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 msgr2=0x555613370050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613370050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 -- 172.21.15.3:0/2875012095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556133e5640 con 0x5556133585d0 2024-09-12T02:19:30.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe710a14640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe70009afb0 tx=0x7fe7000045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f78bfcf034ad2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe6ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.110+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70009f660 con 0x5556133585d0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe700097920 con 0x5556133585d0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe704011dd0 con 0x5556133585d0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7000b4200 con 0x5556133585d0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe715051740 10 client.25334 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556133e68e0 con 0x5556133585d0 2024-09-12T02:19:30.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704011dd0 con 0x5556133585d0 2024-09-12T02:19:30.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe7000bb0b0 con 0x5556133585d0 2024-09-12T02:19:30.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe715051740 10 client.25334 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.111+0000 7fe715051740 10 client.25334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe715051740 20 client.25334 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x5556133e6b70 0x5556133e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe715051740 4 client.25334 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556132c5d70 con 0x5556133e6b70 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x5556133e6b70 0x5556133e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe70bfff640 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x5556133e6b70 0x5556133e8f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e32309e0163235df server_cookie=8a72d9334a5a3b4c in_seq=0 out_seq=0 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe6ebfff640 10 client.25334 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe6ebfff640 1 -- 172.21.15.3:0/2875012095 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556132c5d70 con 0x5556133e6b70 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe6ebfff640 10 client.25334 handle_command_reply: tid=0 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.112+0000 7fe715051740 1 client.25334 shutdown 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 msgr2=0x55561336fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe70009afb0 tx=0x7fe7000045b0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 shutdown_connections 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x5556133e6b70 0x5556133e8f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe6ebfff640 0 client.25334 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55561336bb80 0x555613370590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555613359b40 0x555613370050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 --2- 172.21.15.3:0/2875012095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5556133585d0 0x55561336fb10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 >> 172.21.15.3:0/2875012095 conn(0x555613321310 msgr2=0x55561336a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 shutdown_connections 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 1 -- 172.21.15.3:0/2875012095 wait complete. 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.113+0000 7fe715051740 20 client.25334 trim_cache size 0 max 16384 2024-09-12T02:19:30.118 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 msgr2=0x7fe6fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 0x7fe6fc0a50d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe704005c40 tx=0x7fe704004ab0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 msgr2=0x7fe70c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d950 tx=0x7fe70000de10 comp rx=0 tx=0).stop 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 shutdown_connections 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6fc0a2c10 0x7fe6fc0a50d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe70c10c2f0 0x7fe70c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe70c1037a0 0x7fe70c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 --2- 172.21.15.3:0/1785645913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe70c102df0 0x7fe70c100a50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.114+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 >> 172.21.15.3:0/1785645913 conn(0x7fe70c0fc7d0 msgr2=0x7fe70c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.115+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 shutdown_connections 2024-09-12T02:19:30.120 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:30.115+0000 7fe712c78640 1 -- 172.21.15.3:0/1785645913 wait complete. 2024-09-12T02:19:30.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:30 smithi003 ceph-mon[31476]: pgmap v369: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T02:19:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:30 smithi092 ceph-mon[36089]: pgmap v369: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T02:19:30.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:30 smithi026 ceph-mon[35496]: pgmap v369: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T02:19:30.794 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:19:31.078 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:31.361 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:19:31.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 -- 172.21.15.3:0/1121800254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac106f20 msgr2=0x7f68ac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 --2- 172.21.15.3:0/1121800254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac106f20 0x7f68ac10eaa0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009ca0 tx=0x7f68a402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 -- 172.21.15.3:0/1121800254 shutdown_connections 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 --2- 172.21.15.3:0/1121800254 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac106f20 0x7f68ac10eaa0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 --2- 172.21.15.3:0/1121800254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac1027a0 0x7f68ac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 --2- 172.21.15.3:0/1121800254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 -- 172.21.15.3:0/1121800254 >> 172.21.15.3:0/1121800254 conn(0x7f68ac0fd8f0 msgr2=0x7f68ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 -- 172.21.15.3:0/1121800254 shutdown_connections 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.588+0000 7f68b44bf640 1 -- 172.21.15.3:0/1121800254 wait complete. 2024-09-12T02:19:31.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 Processor -- start 2024-09-12T02:19:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 -- start start 2024-09-12T02:19:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 0x7f68ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac106f20 0x7f68ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac111f80 con 0x7f68ac1027a0 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac111e00 con 0x7f68ac101df0 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.589+0000 7f68b44bf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac112100 con 0x7f68ac106f20 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b1a5a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 0x7f68ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:39998/0 (socket says 172.21.15.3:39998) 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 -- 172.21.15.3:0/699812627 learned_addr learned my addr 172.21.15.3:0/699812627 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac106f20 0x7f68ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 -- 172.21.15.3:0/699812627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac106f20 msgr2=0x7f68ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac106f20 0x7f68ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 -- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 msgr2=0x7f68ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 0x7f68ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b225b640 1 -- 172.21.15.3:0/699812627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ac1a6910 con 0x7f68ac101df0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.590+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 0x7f68ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68b225b640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e9f0 tx=0x7f68a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68a37fe640 1 -- 172.21.15.3:0/699812627 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a800cd80 con 0x7f68ac101df0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68a37fe640 1 -- 172.21.15.3:0/699812627 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a8010910 con 0x7f68ac101df0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68a37fe640 1 -- 172.21.15.3:0/699812627 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8018980 con 0x7f68ac101df0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ac1a6c00 con 0x7f68ac101df0 2024-09-12T02:19:31.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.591+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ac1a7190 con 0x7f68ac101df0 2024-09-12T02:19:31.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.595+0000 7f68a37fe640 1 -- 172.21.15.3:0/699812627 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f68a8010ac0 con 0x7f68ac101df0 2024-09-12T02:19:31.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.595+0000 7f68b6898740 1 Processor -- start 2024-09-12T02:19:31.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.595+0000 7f68b6898740 1 -- start start 2024-09-12T02:19:31.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68b6898740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381228c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68b6898740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x56381228ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68a37fe640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 0x7f68980a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68a37fe640 1 -- 172.21.15.3:0/699812627 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f68a80cf4c0 con 0x7f68ac101df0 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68b2a5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381228c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.596+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 0x7f68980a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b6898740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563812297370 0x563812299760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b6898740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5638121f22f0 con 0x56381228c5d0 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b6898740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x563812199cf0 con 0x563812297370 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b6898740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5638121ee2b0 con 0x56381228db40 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b2a5c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381228c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:54820/0 (socket says 172.21.15.3:54820) 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b2a5c640 1 -- 172.21.15.3:0/2152390061 learned_addr learned my addr 172.21.15.3:0/2152390061 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x56381228ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563812297370 0x563812299760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 -- 172.21.15.3:0/2152390061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 msgr2=0x56381228c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381228c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 -- 172.21.15.3:0/2152390061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563812297370 msgr2=0x563812299760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563812297370 0x563812299760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b225b640 1 -- 172.21.15.3:0/2152390061 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56381217e3a0 con 0x56381228c5d0 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.597+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 0x7f68980a51d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1087f0 tx=0x7f689c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.598+0000 7f68b225b640 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x56381228ca10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a80cce00 tx=0x7f68a809bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b8d6f63f7004ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.598+0000 7f68a0ff9640 1 -- 172.21.15.3:0/2152390061 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8098270 con 0x56381228c5d0 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.598+0000 7f68a0ff9640 1 -- 172.21.15.3:0/2152390061 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80cd050 con 0x56381228c5d0 2024-09-12T02:19:31.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.598+0000 7f68a0ff9640 1 -- 172.21.15.3:0/2152390061 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8010430 con 0x56381228c5d0 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 -- 172.21.15.3:0/2152390061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 msgr2=0x56381228ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x56381228ca10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f68a80cce00 tx=0x7f68a809bb00 comp rx=0 tx=0).stop 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 -- 172.21.15.3:0/2152390061 shutdown_connections 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563812297370 0x563812299760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x56381228ca10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 --2- 172.21.15.3:0/2152390061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381228c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 -- 172.21.15.3:0/2152390061 >> 172.21.15.3:0/2152390061 conn(0x563812255310 msgr2=0x563812288c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 -- 172.21.15.3:0/2152390061 shutdown_connections 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.599+0000 7f68b6898740 1 -- 172.21.15.3:0/2152390061 wait complete. 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 -- start start 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 Processor -- start 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228c5d0 0x563812294100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x563812294b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56381229a070 con 0x563812297370 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5638121ad060 con 0x56381228db40 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.600+0000 7f68b6898740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56381229a1f0 con 0x56381228c5d0 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b6898740 10 client.0 _ll_register_callbacks cb 0x563811f008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40018/0 (socket says 172.21.15.3:40018) 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 -- 172.21.15.3:0/519809083 learned_addr learned my addr 172.21.15.3:0/519809083 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b225b640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228c5d0 0x563812294100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x563812294b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 -- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228c5d0 msgr2=0x563812294100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228c5d0 0x563812294100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 -- 172.21.15.3:0/519809083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 msgr2=0x563812294b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x563812294b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 -- 172.21.15.3:0/519809083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563811f633c0 con 0x56381228db40 2024-09-12T02:19:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x563812294b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.601+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f689c004610 tx=0x7f689c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b82788af7a1316 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.602+0000 7f6895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.602+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00e8b0 con 0x56381228db40 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a80cd050 con 0x56381228db40 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f689c00ea50 con 0x56381228db40 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f689c00ec50 con 0x56381228db40 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 10 client.25346 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638122a5c40 con 0x56381228db40 2024-09-12T02:19:31.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a80cd050 con 0x56381228db40 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f689c040050 con 0x56381228db40 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 10 client.25346 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 20 client.25346 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5638122a74c0 0x5638122a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 4 client.25346 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.603+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638121ff860 con 0x5638122a74c0 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.604+0000 7f68b225b640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5638122a74c0 0x5638122a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.605+0000 7f68b225b640 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5638122a74c0 0x5638122a9900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9392841ea1f82f3b server_cookie=e57824644f5d6e6b in_seq=0 out_seq=0 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.605+0000 7f6895ffb640 10 client.25346 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.605+0000 7f6895ffb640 1 -- 172.21.15.3:0/519809083 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638121ff860 con 0x5638122a74c0 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.605+0000 7f6895ffb640 10 client.25346 handle_command_reply: tid=0 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.605+0000 7f68b6898740 1 client.25346 shutdown 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.606+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 msgr2=0x563812294640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.606+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f689c004610 tx=0x7f689c004730 comp rx=0 tx=0).stop 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 shutdown_connections 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5638122a74c0 0x5638122a9900 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x563812294b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x563812294640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 --2- 172.21.15.3:0/519809083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228c5d0 0x563812294100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 >> 172.21.15.3:0/519809083 conn(0x563812255310 msgr2=0x56381228e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f6895ffb640 0 client.25346 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 shutdown_connections 2024-09-12T02:19:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.607+0000 7f68b6898740 1 -- 172.21.15.3:0/519809083 wait complete. 2024-09-12T02:19:31.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.608+0000 7f68b6898740 20 client.25346 trim_cache size 0 max 16384 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.632+0000 7f68b6898740 1 Processor -- start 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.632+0000 7f68b6898740 1 -- start start 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.632+0000 7f68b6898740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b6898740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x5638122af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b6898740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563812297370 0x5638122af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b6898740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1ab510 con 0x56381228c5d0 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b6898740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1ad050 con 0x56381228db40 2024-09-12T02:19:31.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b6898740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1abbe0 con 0x563812297370 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45554/0 (socket says 172.21.15.3:45554) 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 -- 172.21.15.3:0/1899480324 learned_addr learned my addr 172.21.15.3:0/1899480324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 -- 172.21.15.3:0/1899480324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563812297370 msgr2=0x5638122af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563812297370 0x5638122af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563812297370 0x5638122af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 -- 172.21.15.3:0/1899480324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 msgr2=0x5638122af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x5638122af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.633+0000 7f68b225b640 1 -- 172.21.15.3:0/1899480324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563812292650 con 0x56381228c5d0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.634+0000 7f68b225b640 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f68a809b900 tx=0x7f68a8018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4018228fe0aab452 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.634+0000 7f6897fff640 1 -- 172.21.15.3:0/1899480324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8098270 con 0x56381228c5d0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.634+0000 7f6897fff640 1 -- 172.21.15.3:0/1899480324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a809b1c0 con 0x56381228c5d0 2024-09-12T02:19:31.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.634+0000 7f6897fff640 1 -- 172.21.15.3:0/1899480324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a80ba570 con 0x56381228c5d0 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 -- 172.21.15.3:0/1899480324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 msgr2=0x563812288e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f68a809b900 tx=0x7f68a8018420 comp rx=0 tx=0).stop 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 -- 172.21.15.3:0/1899480324 shutdown_connections 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563812297370 0x5638122af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228db40 0x5638122af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 --2- 172.21.15.3:0/1899480324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56381228c5d0 0x563812288e10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.635+0000 7f68b6898740 1 -- 172.21.15.3:0/1899480324 >> 172.21.15.3:0/1899480324 conn(0x563812255310 msgr2=0x563812288700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.636+0000 7f68b6898740 1 -- 172.21.15.3:0/1899480324 shutdown_connections 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.636+0000 7f68b6898740 1 -- 172.21.15.3:0/1899480324 wait complete. 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.636+0000 7f68b6898740 1 -- start start 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.636+0000 7f68b6898740 1 Processor -- start 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381230a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x56381229aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1ac980 con 0x563812297370 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1ac2b0 con 0x56381228c5d0 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f68ac1a94f0 con 0x56381228db40 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b6898740 10 client.0 _ll_register_callbacks cb 0x7f689c004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40038/0 (socket says 172.21.15.3:40038) 2024-09-12T02:19:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 -- 172.21.15.3:0/1375831778 learned_addr learned my addr 172.21.15.3:0/1375831778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:31.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381230a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x56381229aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 -- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 msgr2=0x56381230a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381230a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 -- 172.21.15.3:0/1375831778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 msgr2=0x56381229aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x56381229aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b225b640 1 -- 172.21.15.3:0/1375831778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381229b150 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.637+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381230a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f68b225b640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68a809bb90 tx=0x7f68a80cce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bc9ee381d8181a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f6895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a807e070 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a809b1c0 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f68a80cd050 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68a80cf070 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f68b2a5c640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x56381229aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f68b6898740 10 client.25349 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563812311cf0 con 0x56381228c5d0 2024-09-12T02:19:31.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.638+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a809b1c0 con 0x56381228c5d0 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68a8097950 con 0x56381228c5d0 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 10 client.25349 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 10 client.25349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 20 client.25349 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x563812311f20 0x563812314310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 4 client.25349 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638121f9d70 con 0x563812311f20 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x563812311f20 0x563812314310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.639+0000 7f6895ffb640 10 client.25349 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f68b1a5a640 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x563812311f20 0x563812314310 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59511efe54818e3 server_cookie=801ba25c2c14518b in_seq=0 out_seq=0 2024-09-12T02:19:31.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f6895ffb640 1 -- 172.21.15.3:0/1375831778 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638121f9d70 con 0x563812311f20 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f6895ffb640 10 client.25349 handle_command_reply: tid=0 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f68b6898740 1 client.25349 shutdown 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 msgr2=0x56381230a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.640+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68a809bb90 tx=0x7f68a80cce50 comp rx=0 tx=0).stop 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 shutdown_connections 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x563812311f20 0x563812314310 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563812297370 0x56381229aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56381228db40 0x56381230a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 --2- 172.21.15.3:0/1375831778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56381228c5d0 0x56381230a490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 >> 172.21.15.3:0/1375831778 conn(0x563812255310 msgr2=0x56381228e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f6895ffb640 0 client.25349 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 shutdown_connections 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 1 -- 172.21.15.3:0/1375831778 wait complete. 2024-09-12T02:19:31.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.641+0000 7f68b6898740 20 client.25349 trim_cache size 0 max 16384 2024-09-12T02:19:31.648 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:31.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 msgr2=0x7f68980a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 0x7f68980a51d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f68ac1087f0 tx=0x7f689c005f90 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 msgr2=0x7f68ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e9f0 tx=0x7f68a800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 shutdown_connections 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f68980a2d10 0x7f68980a51d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68ac106f20 0x7f68ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f68ac1027a0 0x7f68ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 --2- 172.21.15.3:0/699812627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68ac101df0 0x7f68ac0744f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.642+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 >> 172.21.15.3:0/699812627 conn(0x7f68ac0fd8f0 msgr2=0x7f68ac0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.643+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 shutdown_connections 2024-09-12T02:19:31.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:31.643+0000 7f68b44bf640 1 -- 172.21.15.3:0/699812627 wait complete. 2024-09-12T02:19:31.908 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:32.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:32 smithi003 ceph-mon[31476]: pgmap v370: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-12T02:19:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:32 smithi092 ceph-mon[36089]: pgmap v370: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-12T02:19:32.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:32 smithi026 ceph-mon[35496]: pgmap v370: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 13 op/s 2024-09-12T02:19:33.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 -- 172.21.15.3:0/1757596907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8102d20 msgr2=0x7f11e8103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1757596907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8102d20 0x7f11e8103100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009900 tx=0x7f11dc02a480 comp rx=0 tx=0).stop 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 -- 172.21.15.3:0/1757596907 shutdown_connections 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1757596907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e810c150 0x7f11e810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1757596907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e8103640 0x7f11e810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1757596907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8102d20 0x7f11e8103100 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.124+0000 7f11eeee1640 1 -- 172.21.15.3:0/1757596907 >> 172.21.15.3:0/1757596907 conn(0x7f11e80fc780 msgr2=0x7f11e80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:33.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.125+0000 7f11eeee1640 1 -- 172.21.15.3:0/1757596907 shutdown_connections 2024-09-12T02:19:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.125+0000 7f11eeee1640 1 -- 172.21.15.3:0/1757596907 wait complete. 2024-09-12T02:19:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.126+0000 7f11eeee1640 1 Processor -- start 2024-09-12T02:19:33.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.127+0000 7f11eeee1640 1 -- start start 2024-09-12T02:19:33.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.127+0000 7f11eeee1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.127+0000 7f11eeee1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 0x7f11e81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.127+0000 7f11eeee1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e810c150 0x7f11e819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11eeee1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8111f50 con 0x7f11e8103640 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11eeee1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8111dd0 con 0x7f11e810c150 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:54860/0 (socket says 172.21.15.3:54860) 2024-09-12T02:19:33.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 -- 172.21.15.3:0/1485024055 learned_addr learned my addr 172.21.15.3:0/1485024055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:33.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f11e81120d0 con 0x7f11e8102d20 2024-09-12T02:19:33.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ed47e640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e810c150 0x7f11e819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11e7fff640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 0x7f11e81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 -- 172.21.15.3:0/1485024055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e810c150 msgr2=0x7f11e819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e810c150 0x7f11e819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.128+0000 7f11ecc7d640 1 -- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 msgr2=0x7f11e81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11ecc7d640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 0x7f11e81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11ecc7d640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819d320 con 0x7f11e8102d20 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11e7fff640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 0x7f11e81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11ecc7d640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11dc005ab0 tx=0x7f11dc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:33.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc03f030 con 0x7f11e8102d20 2024-09-12T02:19:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819d5b0 con 0x7f11e8102d20 2024-09-12T02:19:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc003ce0 con 0x7f11e8102d20 2024-09-12T02:19:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.129+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc035070 con 0x7f11e8102d20 2024-09-12T02:19:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.130+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e81a4ef0 con 0x7f11e8102d20 2024-09-12T02:19:33.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.131+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e806b070 con 0x7f11e8102d20 2024-09-12T02:19:33.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.133+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f11dc042030 con 0x7f11e8102d20 2024-09-12T02:19:33.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.134+0000 7f11e5ffb640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 0x7f11d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:33.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.134+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f11dc0efa60 con 0x7f11e8102d20 2024-09-12T02:19:33.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.134+0000 7f11e7fff640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 0x7f11d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:33.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.135+0000 7f11e7fff640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 0x7f11d80a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005e10 tx=0x7f11d4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:33.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.139+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11dc0bc850 con 0x7f11e8102d20 2024-09-12T02:19:33.425 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:19:33.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:19:33.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.568+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e81008b0 con 0x7f11e8102d20 2024-09-12T02:19:33.570 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:19:33.570 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:19:33.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.569+0000 7f11e5ffb640 1 -- 172.21.15.3:0/1485024055 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7f11dc0bc150 con 0x7f11e8102d20 2024-09-12T02:19:33.571 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:19:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.572+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 msgr2=0x7f11d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.572+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 0x7f11d80a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005e10 tx=0x7f11d4005da0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.572+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 msgr2=0x7f11e8198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.572+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11dc005ab0 tx=0x7f11dc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 shutdown_connections 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f11d80a2c90 0x7f11d80a5150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11e810c150 0x7f11e819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11e8103640 0x7f11e81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 --2- 172.21.15.3:0/1485024055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8102d20 0x7f11e8198370 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11dc005ab0 tx=0x7f11dc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 >> 172.21.15.3:0/1485024055 conn(0x7f11e80fc780 msgr2=0x7f11e80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.573+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 shutdown_connections 2024-09-12T02:19:33.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:33.574+0000 7f11eeee1640 1 -- 172.21.15.3:0/1485024055 wait complete. 2024-09-12T02:19:34.241 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db --set-id ddfbd753 --release --await' 2024-09-12T02:19:34.242 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db --set-id ddfbd753 --release --await 2024-09-12T02:19:34.257 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:34 smithi003 ceph-mon[31476]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:34 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1485024055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:34.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:34 smithi092 ceph-mon[36089]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:34.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:34 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1485024055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:34.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:34 smithi026 ceph-mon[35496]: pgmap v371: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:34.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:34 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1485024055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 -- 172.21.15.3:0/840424132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30101de0 msgr2=0x7f5e301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 --2- 172.21.15.3:0/840424132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30101de0 0x7f5e301021c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009900 tx=0x7f5e2402a410 comp rx=0 tx=0).stop 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 -- 172.21.15.3:0/840424132 shutdown_connections 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 --2- 172.21.15.3:0/840424132 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30106f10 0x7f5e3010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 --2- 172.21.15.3:0/840424132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 0x7f5e301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 --2- 172.21.15.3:0/840424132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30101de0 0x7f5e301021c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.779+0000 7f5e36f46640 1 -- 172.21.15.3:0/840424132 >> 172.21.15.3:0/840424132 conn(0x7f5e300fd8c0 msgr2=0x7f5e300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.780+0000 7f5e36f46640 1 -- 172.21.15.3:0/840424132 shutdown_connections 2024-09-12T02:19:34.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.780+0000 7f5e36f46640 1 -- 172.21.15.3:0/840424132 wait complete. 2024-09-12T02:19:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 Processor -- start 2024-09-12T02:19:34.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 -- start start 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 0x7f5e30198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 0x7f5e30198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30111f70 con 0x7f5e30106f10 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.781+0000 7f5e36f46640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30111df0 con 0x7f5e30102790 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45622/0 (socket says 172.21.15.3:45622) 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 -- 172.21.15.3:0/611465503 learned_addr learned my addr 172.21.15.3:0/611465503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 0x7f5e30198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 0x7f5e30198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301120f0 con 0x7f5e30101de0 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 -- 172.21.15.3:0/611465503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 msgr2=0x7f5e30198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 0x7f5e30198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 -- 172.21.15.3:0/611465503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 msgr2=0x7f5e30198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 0x7f5e30198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e354e3640 1 -- 172.21.15.3:0/611465503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3019d3e0 con 0x7f5e30106f10 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.782+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 0x7f5e30198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e354e3640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800eaa0 tx=0x7f5e2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e2dffb640 1 -- 172.21.15.3:0/611465503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cdb0 con 0x7f5e30106f10 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e2dffb640 1 -- 172.21.15.3:0/611465503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e28005020 con 0x7f5e30106f10 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3019d6d0 con 0x7f5e30106f10 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e2dffb640 1 -- 172.21.15.3:0/611465503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280116d0 con 0x7f5e30106f10 2024-09-12T02:19:34.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.783+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e301a4fb0 con 0x7f5e30106f10 2024-09-12T02:19:34.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.786+0000 7f5e3931f740 1 Processor -- start 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e2dffb640 1 -- 172.21.15.3:0/611465503 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5e280054d0 con 0x7f5e30106f10 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 -- start start 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcbf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcb96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddc15280 con 0x55b3ddcb9d60 2024-09-12T02:19:34.787 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddbbcb10 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.787+0000 7f5e3931f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddc11240 con 0x55b3ddcbeda0 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45636/0 (socket says 172.21.15.3:45636) 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 -- 172.21.15.3:0/33699193 learned_addr learned my addr 172.21.15.3:0/33699193 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcb96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcbf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2dffb640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 0x7f5e200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2dffb640 1 -- 172.21.15.3:0/611465503 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5e280cf320 con 0x7f5e30106f10 2024-09-12T02:19:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e354e3640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 0x7f5e200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 -- 172.21.15.3:0/33699193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbeda0 msgr2=0x55b3ddcbf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcbf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 -- 172.21.15.3:0/33699193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbf6c0 msgr2=0x55b3ddcb96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.788+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcb96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.789+0000 7f5e2ffff640 1 -- 172.21.15.3:0/33699193 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ddbc91f0 con 0x55b3ddcb9d60 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.789+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0127c0 tx=0x7f5e1c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6d32afa2827fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.789+0000 7f5e354e3640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 0x7f5e200a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fd0 tx=0x7f5e28009f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.789+0000 7f5e1affd640 1 -- 172.21.15.3:0/33699193 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c012e00 con 0x55b3ddcb9d60 2024-09-12T02:19:34.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.790+0000 7f5e1affd640 1 -- 172.21.15.3:0/33699193 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1c010c50 con 0x55b3ddcb9d60 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.790+0000 7f5e3931f740 1 -- 172.21.15.3:0/33699193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 msgr2=0x55b3ddcba1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.790+0000 7f5e3931f740 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0127c0 tx=0x7f5e1c00b7c0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 -- 172.21.15.3:0/33699193 shutdown_connections 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcb9d60 0x55b3ddcba1e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcb96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 --2- 172.21.15.3:0/33699193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcbf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 -- 172.21.15.3:0/33699193 >> 172.21.15.3:0/33699193 conn(0x55b3ddcabba0 msgr2=0x55b3ddcabf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 -- 172.21.15.3:0/33699193 shutdown_connections 2024-09-12T02:19:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 -- 172.21.15.3:0/33699193 wait complete. 2024-09-12T02:19:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 -- start start 2024-09-12T02:19:34.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.791+0000 7f5e3931f740 1 Processor -- start 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcc2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddcc6180 con 0x55b3ddcc3610 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddbcff10 con 0x55b3ddcbeda0 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b3ddcc6300 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.792+0000 7f5e3931f740 10 client.0 _ll_register_callbacks cb 0x55b3ddb98a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:54908/0 (socket says 172.21.15.3:54908) 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 -- 172.21.15.3:0/2778400013 learned_addr learned my addr 172.21.15.3:0/2778400013 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 -- 172.21.15.3:0/2778400013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 msgr2=0x55b3ddcc2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcc2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 -- 172.21.15.3:0/2778400013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 msgr2=0x55b3ddccf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 -- 172.21.15.3:0/2778400013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3dd9866a0 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009900 tx=0x7f5e240318d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a234b7cbdd4cf6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.793+0000 7f5e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24035040 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5e24004240 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c010c50 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e24004440 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e3931f740 10 client.25444 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:34.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3ddcd23b0 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.794+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e24005510 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5e240035f0 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 10 client.25444 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 20 client.25444 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddcd38c0 0x55b3ddcd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 4 client.25444 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e1c010c50 con 0x55b3ddcd38c0 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.795+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddcd38c0 0x55b3ddcd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.796+0000 7f5e0ffff640 10 client.25444 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.796+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddcd38c0 0x55b3ddcd5d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbff48cb7a0f01f2 server_cookie=ab95db3748814017 in_seq=0 out_seq=0 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.796+0000 7f5e0ffff640 1 -- 172.21.15.3:0/2778400013 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5e1c010c50 con 0x55b3ddcd38c0 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.796+0000 7f5e0ffff640 10 client.25444 handle_command_reply: tid=0 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.796+0000 7f5e3931f740 1 client.25444 shutdown 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 msgr2=0x55b3ddcc30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009900 tx=0x7f5e240318d0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 shutdown_connections 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddcd38c0 0x55b3ddcd5d00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc30d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e0ffff640 0 client.25444 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 --2- 172.21.15.3:0/2778400013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcc2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 >> 172.21.15.3:0/2778400013 conn(0x55b3ddcabba0 msgr2=0x55b3ddcab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 shutdown_connections 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 1 -- 172.21.15.3:0/2778400013 wait complete. 2024-09-12T02:19:34.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.797+0000 7f5e3931f740 20 client.25444 trim_cache size 0 max 16384 2024-09-12T02:19:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 Processor -- start 2024-09-12T02:19:34.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 -- start start 2024-09-12T02:19:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301ab470 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301acfb0 con 0x55b3ddcbeda0 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e3931f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301abb40 con 0x55b3ddcc3610 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.829+0000 7f5e34ce2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40102/0 (socket says 172.21.15.3:40102) 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e34ce2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45658/0 (socket says 172.21.15.3:45658) 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3694342431 learned_addr learned my addr 172.21.15.3:0/3694342431 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3694342431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcc3610 msgr2=0x55b3ddccf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3694342431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 msgr2=0x55b3ddcbe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3694342431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ddca7060 con 0x55b3ddcbeda0 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.830+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.831+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c01e7c0 tx=0x7f5e1c01b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11bfb64fbbb5f6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.831+0000 7f5e1a7fc640 1 -- 172.21.15.3:0/3694342431 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c010e80 con 0x55b3ddcbeda0 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.831+0000 7f5e1a7fc640 1 -- 172.21.15.3:0/3694342431 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1c010c20 con 0x55b3ddcbeda0 2024-09-12T02:19:34.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.831+0000 7f5e1a7fc640 1 -- 172.21.15.3:0/3694342431 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c005040 con 0x55b3ddcbeda0 2024-09-12T02:19:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.832+0000 7f5e3931f740 1 -- 172.21.15.3:0/3694342431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 msgr2=0x55b3ddcb96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.832+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c01e7c0 tx=0x7f5e1c01b040 comp rx=0 tx=0).stop 2024-09-12T02:19:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.832+0000 7f5e3931f740 1 -- 172.21.15.3:0/3694342431 shutdown_connections 2024-09-12T02:19:34.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.832+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcc3610 0x55b3ddccf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.832+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcbe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.833+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3694342431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcbeda0 0x55b3ddcb96f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.833+0000 7f5e3931f740 1 -- 172.21.15.3:0/3694342431 >> 172.21.15.3:0/3694342431 conn(0x55b3ddcabba0 msgr2=0x55b3ddcaaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.833+0000 7f5e3931f740 1 -- 172.21.15.3:0/3694342431 shutdown_connections 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.833+0000 7f5e3931f740 1 -- 172.21.15.3:0/3694342431 wait complete. 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.834+0000 7f5e3931f740 1 -- start start 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.834+0000 7f5e3931f740 1 Processor -- start 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcc3610 0x55b3ddcc6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e2ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45666/0 (socket says 172.21.15.3:45666) 2024-09-12T02:19:34.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301ac8e0 con 0x55b3ddcbeda0 2024-09-12T02:19:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3452555297 learned_addr learned my addr 172.21.15.3:0/3452555297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301ac210 con 0x55b3ddcc3610 2024-09-12T02:19:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5e301a9450 con 0x55b3ddcbf6c0 2024-09-12T02:19:34.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.835+0000 7f5e3931f740 10 client.0 _ll_register_callbacks cb 0x7f5e24005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:34.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.836+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 msgr2=0x55b3ddcc65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.836+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.836+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3452555297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcc3610 msgr2=0x55b3ddcc6ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.836+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcc3610 0x55b3ddcc6ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.836+0000 7f5e2ffff640 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ddcc70d0 con 0x55b3ddcbeda0 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.837+0000 7f5e2ffff640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0127c0 tx=0x7f5e1c012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ccdf12ad61fd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.837+0000 7f5e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.837+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c038030 con 0x55b3ddcbeda0 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.837+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5e1c02e070 con 0x55b3ddcbeda0 2024-09-12T02:19:34.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.837+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c010c20 con 0x55b3ddcbeda0 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.838+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c043050 con 0x55b3ddcbeda0 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.838+0000 7f5e3931f740 10 client.15813 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.838+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3ddd3f3e0 con 0x55b3ddcbeda0 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.838+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e1c00bad0 con 0x55b3ddcbeda0 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.838+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5e1c046020 con 0x55b3ddcbeda0 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.839+0000 7f5e3931f740 10 client.15813 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:34.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.839+0000 7f5e3931f740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.839+0000 7f5e3931f740 20 client.15813 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.839+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddd3f7b0 0x55b3ddd41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.839+0000 7f5e3931f740 4 client.15813 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:19:34.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.840+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddd3f7b0 0x55b3ddd41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.840+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5e1c010c20 con 0x55b3ddd3f7b0 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.841+0000 7f5e0ffff640 10 client.15813 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.841+0000 7f5e34ce2640 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddd3f7b0 0x55b3ddd41ba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb14b4258c59984 server_cookie=3a2615624bf454eb in_seq=0 out_seq=0 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.842+0000 7f5e0ffff640 1 -- 172.21.15.3:0/3452555297 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5e1c010c20 con 0x55b3ddd3f7b0 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.842+0000 7f5e0ffff640 10 client.15813 handle_command_reply: tid=0 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.842+0000 7f5e3931f740 1 client.15813 shutdown 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.842+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 msgr2=0x55b3ddccbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.842+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0127c0 tx=0x7f5e1c012500 comp rx=0 tx=0).stop 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 shutdown_connections 2024-09-12T02:19:34.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b3ddd3f7b0 0x55b3ddd41ba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ddcc3610 0x55b3ddcc6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e0ffff640 0 client.15813 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ddcbf6c0 0x55b3ddcc65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 --2- 172.21.15.3:0/3452555297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b3ddcbeda0 0x55b3ddccbb70 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0127c0 tx=0x7f5e1c012500 comp rx=0 tx=0).stop 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.843+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 >> 172.21.15.3:0/3452555297 conn(0x55b3ddcabba0 msgr2=0x55b3ddcab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.844+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 shutdown_connections 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.844+0000 7f5e3931f740 1 -- 172.21.15.3:0/3452555297 wait complete. 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.844+0000 7f5e3931f740 20 client.15813 trim_cache size 0 max 16384 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:34.845 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-12T02:19:34.846 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "2db07bef-3a63-4fd9-8c63-ebcca0dc4464", 2024-09-12T02:19:34.846 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-12T02:19:34.846 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 msgr2=0x7f5e200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 0x7f5e200a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009fd0 tx=0x7f5e28009f60 comp rx=0 tx=0).stop 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 msgr2=0x7f5e3019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800eaa0 tx=0x7f5e2800ef60 comp rx=0 tx=0).stop 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 shutdown_connections 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5e200a2c90 0x7f5e200a5150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e30106f10 0x7f5e3019cd00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e30102790 0x7f5e30198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 --2- 172.21.15.3:0/611465503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e30101de0 0x7f5e30198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 >> 172.21.15.3:0/611465503 conn(0x7f5e300fd8c0 msgr2=0x7f5e300fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 shutdown_connections 2024-09-12T02:19:34.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:34.846+0000 7f5e36f46640 1 -- 172.21.15.3:0/611465503 wait complete. 2024-09-12T02:19:34.864 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 -- 172.21.15.3:0/2039681624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 msgr2=0x7fc6241020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2039681624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc6241020b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc610009900 tx=0x7fc61002a410 comp rx=0 tx=0).stop 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 -- 172.21.15.3:0/2039681624 shutdown_connections 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2039681624 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6241025f0 0x7fc624108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2039681624 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc6241020b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2039681624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc624100a40 0x7fc624100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 -- 172.21.15.3:0/2039681624 >> 172.21.15.3:0/2039681624 conn(0x7fc6240fc1c0 msgr2=0x7fc6240fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 -- 172.21.15.3:0/2039681624 shutdown_connections 2024-09-12T02:19:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.931+0000 7fc62af1a640 1 -- 172.21.15.3:0/2039681624 wait complete. 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 Processor -- start 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 -- start start 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 0x7fc62406a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc62419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241025f0 0x7fc62419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410c710 con 0x7fc624101c30 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410c590 con 0x7fc6241025f0 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc62af1a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc62410c890 con 0x7fc624100a40 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc623fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc62419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc628cb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 0x7fc62406a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc623fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc62419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45678/0 (socket says 172.21.15.3:45678) 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.932+0000 7fc623fff640 1 -- 172.21.15.3:0/2360802020 learned_addr learned my addr 172.21.15.3:0/2360802020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc6294b7640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241025f0 0x7fc62419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 -- 172.21.15.3:0/2360802020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241025f0 msgr2=0x7fc62419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241025f0 0x7fc62419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 -- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 msgr2=0x7fc62419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc62419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 -- 172.21.15.3:0/2360802020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62419b6a0 con 0x7fc624100a40 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 0x7fc62406a5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc61800d6a0 tx=0x7fc61800db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc621ffb640 1 -- 172.21.15.3:0/2360802020 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618017070 con 0x7fc624100a40 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62419b990 con 0x7fc624100a40 2024-09-12T02:19:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc621ffb640 1 -- 172.21.15.3:0/2360802020 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc618009d00 con 0x7fc624100a40 2024-09-12T02:19:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.933+0000 7fc621ffb640 1 -- 172.21.15.3:0/2360802020 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618003c60 con 0x7fc624100a40 2024-09-12T02:19:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.934+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62419bf30 con 0x7fc624100a40 2024-09-12T02:19:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.936+0000 7fc621ffb640 1 -- 172.21.15.3:0/2360802020 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc618009820 con 0x7fc624100a40 2024-09-12T02:19:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.936+0000 7fc621ffb640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 0x7fc6140a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc621ffb640 1 -- 172.21.15.3:0/2360802020 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc6180ceb60 con 0x7fc624100a40 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc623fff640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 0x7fc6140a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 Processor -- start 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 -- start start 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce66742d0 0x55bce6678d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce667b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bce65d72f0 con 0x55bce6673990 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bce657f270 con 0x55bce6679360 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc62dcae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bce65d32b0 con 0x55bce66742d0 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc628cb6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc628cb6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45686/0 (socket says 172.21.15.3:45686) 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.937+0000 7fc628cb6640 1 -- 172.21.15.3:0/3506178122 learned_addr learned my addr 172.21.15.3:0/3506178122 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc623fff640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 0x7fc6140a51d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc62406a330 tx=0x7fc61000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 -- 172.21.15.3:0/3506178122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce66742d0 msgr2=0x55bce6678d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc6294b7640 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce667b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce66742d0 0x55bce6678d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 -- 172.21.15.3:0/3506178122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 msgr2=0x55bce667b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce667b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 -- 172.21.15.3:0/3506178122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce657e340 con 0x55bce6673990 2024-09-12T02:19:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc628cb6640 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc61809b180 tx=0x7fc61809ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3d46177094a5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc60effd640 1 -- 172.21.15.3:0/3506178122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618020040 con 0x55bce6673990 2024-09-12T02:19:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc60effd640 1 -- 172.21.15.3:0/3506178122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6180a3e60 con 0x55bce6673990 2024-09-12T02:19:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.938+0000 7fc60effd640 1 -- 172.21.15.3:0/3506178122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180cc6f0 con 0x55bce6673990 2024-09-12T02:19:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 -- 172.21.15.3:0/3506178122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 msgr2=0x55bce6673d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc61809b180 tx=0x7fc61809ac80 comp rx=0 tx=0).stop 2024-09-12T02:19:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 -- 172.21.15.3:0/3506178122 shutdown_connections 2024-09-12T02:19:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce667b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce66742d0 0x55bce6678d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 --2- 172.21.15.3:0/3506178122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6673990 0x55bce6673d90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 -- 172.21.15.3:0/3506178122 >> 172.21.15.3:0/3506178122 conn(0x55bce666f000 msgr2=0x55bce666f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 -- 172.21.15.3:0/3506178122 shutdown_connections 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.939+0000 7fc62dcae740 1 -- 172.21.15.3:0/3506178122 wait complete. 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.940+0000 7fc62dcae740 1 -- start start 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.940+0000 7fc62dcae740 1 Processor -- start 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.940+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce667dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.940+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6685130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc62dcae740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bce667c270 con 0x55bce6679360 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc62dcae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bce6560290 con 0x55bce66742d0 2024-09-12T02:19:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc62dcae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bce667c3f0 con 0x55bce6673990 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc62dcae740 10 client.0 _ll_register_callbacks cb 0x55bce6591140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40150/0 (socket says 172.21.15.3:40150) 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 -- 172.21.15.3:0/457213638 learned_addr learned my addr 172.21.15.3:0/457213638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc6294b7640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6685130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 -- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 msgr2=0x55bce667dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc628cb6640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce667dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce667dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 -- 172.21.15.3:0/457213638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 msgr2=0x55bce6685130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6685130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 -- 172.21.15.3:0/457213638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce62ea210 con 0x55bce66742d0 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc6294b7640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6685130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc623fff640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc610005720 tx=0x7fc610000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a342a060496fd54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.941+0000 7fc603fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610035040 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6180a3e60 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc610004460 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc610004870 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc62dcae740 10 client.25373 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bce63b6620 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6180a3e60 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.942+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6100035f0 con 0x55bce66742d0 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 10 client.25373 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 20 client.25373 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce668bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 4 client.25373 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bce65e4a20 con 0x55bce66898e0 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc628cb6640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce668bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc628cb6640 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce668bcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80425ec909e69ee server_cookie=65a74e527f728bdd in_seq=0 out_seq=0 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.943+0000 7fc603fff640 10 client.25373 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc603fff640 1 -- 172.21.15.3:0/457213638 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bce65e4a20 con 0x55bce66898e0 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc603fff640 10 client.25373 handle_command_reply: tid=0 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 client.25373 shutdown 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 msgr2=0x55bce667e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc610005720 tx=0x7fc610000bc0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 shutdown_connections 2024-09-12T02:19:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce668bcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc603fff640 0 client.25373 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6685130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce667e0f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 --2- 172.21.15.3:0/457213638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce667dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.944+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 >> 172.21.15.3:0/457213638 conn(0x55bce666f000 msgr2=0x55bce6670810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.945+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 shutdown_connections 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.945+0000 7fc62dcae740 1 -- 172.21.15.3:0/457213638 wait complete. 2024-09-12T02:19:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.945+0000 7fc62dcae740 20 client.25373 trim_cache size 0 max 16384 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.964+0000 7fc62dcae740 1 Processor -- start 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.964+0000 7fc62dcae740 1 -- start start 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6673d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce666e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241af5f0 con 0x55bce66742d0 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241b1130 con 0x55bce6679360 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc62dcae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241afcc0 con 0x55bce6673990 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45694/0 (socket says 172.21.15.3:45694) 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 -- 172.21.15.3:0/923598575 learned_addr learned my addr 172.21.15.3:0/923598575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc6294b7640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce666e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc628cb6640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6673d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 -- 172.21.15.3:0/923598575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 msgr2=0x55bce6673d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6673d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 -- 172.21.15.3:0/923598575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 msgr2=0x55bce666e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce666e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 -- 172.21.15.3:0/923598575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bce666fc90 con 0x55bce66742d0 2024-09-12T02:19:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc6294b7640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce666e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.965+0000 7fc623fff640 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc610009850 tx=0x7fc610003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98108208678bc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.966+0000 7fc60e7fc640 1 -- 172.21.15.3:0/923598575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610061020 con 0x55bce66742d0 2024-09-12T02:19:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.966+0000 7fc60e7fc640 1 -- 172.21.15.3:0/923598575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61004e070 con 0x55bce66742d0 2024-09-12T02:19:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.966+0000 7fc60e7fc640 1 -- 172.21.15.3:0/923598575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610035040 con 0x55bce66742d0 2024-09-12T02:19:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- 172.21.15.3:0/923598575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 msgr2=0x55bce66850f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc610009850 tx=0x7fc610003f50 comp rx=0 tx=0).stop 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- 172.21.15.3:0/923598575 shutdown_connections 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce6679360 0x55bce666e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce66742d0 0x55bce66850f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 --2- 172.21.15.3:0/923598575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6673d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- 172.21.15.3:0/923598575 >> 172.21.15.3:0/923598575 conn(0x55bce666f000 msgr2=0x55bce666d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- 172.21.15.3:0/923598575 shutdown_connections 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- 172.21.15.3:0/923598575 wait complete. 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 -- start start 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.967+0000 7fc62dcae740 1 Processor -- start 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce6682c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6683c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241b0a60 con 0x55bce6679360 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241b0390 con 0x55bce66742d0 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc6241ad5d0 con 0x55bce6673990 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc62dcae740 10 client.0 _ll_register_callbacks cb 0x7fc610040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc628cb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc628cb6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:54964/0 (socket says 172.21.15.3:54964) 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc628cb6640 1 -- 172.21.15.3:0/2809159078 learned_addr learned my addr 172.21.15.3:0/2809159078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc6294b7640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6683c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc623fff640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce6682c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.968+0000 7fc628cb6640 1 -- 172.21.15.3:0/2809159078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 msgr2=0x55bce6682c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce6682c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc628cb6640 1 -- 172.21.15.3:0/2809159078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 msgr2=0x55bce6683c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6683c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc628cb6640 1 -- 172.21.15.3:0/2809159078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bce6684280 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc6294b7640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6683c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc628cb6640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc61809aef0 tx=0x7fc618005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8816a4226c8b709c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc603fff640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180b9060 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fc618097860 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61004e070 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc618020040 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc62dcae740 10 client.25462 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bce66f4590 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.969+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61004e070 con 0x55bce6673990 2024-09-12T02:19:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6180bb0b0 con 0x55bce6673990 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 10 client.25462 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 20 client.25462 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce66f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 4 client.25462 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bce65ded70 con 0x55bce66898e0 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc623fff640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce66f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc623fff640 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce66f6a20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3734819bd3a98b8 server_cookie=ff04db3cb39b6e5b in_seq=0 out_seq=0 2024-09-12T02:19:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.970+0000 7fc603fff640 10 client.25462 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc603fff640 1 -- 172.21.15.3:0/2809159078 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bce65ded70 con 0x55bce66898e0 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc603fff640 10 client.25462 handle_command_reply: tid=0 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 client.25462 shutdown 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 msgr2=0x55bce6682720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc61809aef0 tx=0x7fc618005bf0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 shutdown_connections 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bce66898e0 0x55bce66f6a20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc603fff640 0 client.25462 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bce6679360 0x55bce6683c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bce66742d0 0x55bce6682c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 --2- 172.21.15.3:0/2809159078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bce6673990 0x55bce6682720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 >> 172.21.15.3:0/2809159078 conn(0x55bce666f000 msgr2=0x55bce666d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 shutdown_connections 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 1 -- 172.21.15.3:0/2809159078 wait complete. 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.974+0000 7fc62dcae740 20 client.25462 trim_cache size 0 max 16384 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:19:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:19:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 msgr2=0x7fc6140a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 0x7fc6140a51d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc62406a330 tx=0x7fc61000b040 comp rx=0 tx=0).stop 2024-09-12T02:19:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 msgr2=0x7fc62406a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 0x7fc62406a5c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc61800d6a0 tx=0x7fc61800db60 comp rx=0 tx=0).stop 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 shutdown_connections 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fc6140a2d10 0x7fc6140a51d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6241025f0 0x7fc62419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc624101c30 0x7fc62419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 --2- 172.21.15.3:0/2360802020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc624100a40 0x7fc62406a5c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 >> 172.21.15.3:0/2360802020 conn(0x7fc6240fc1c0 msgr2=0x7fc624104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 shutdown_connections 2024-09-12T02:19:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:34.975+0000 7fc62af1a640 1 -- 172.21.15.3:0/2360802020 wait complete. 2024-09-12T02:19:35.669 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:19:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 -- 172.21.15.3:0/2240558375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc102db0 msgr2=0x7f87dc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 --2- 172.21.15.3:0/2240558375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc102db0 0x7f87dc103190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009930 tx=0x7f87cc02a420 comp rx=0 tx=0).stop 2024-09-12T02:19:36.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 -- 172.21.15.3:0/2240558375 shutdown_connections 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 --2- 172.21.15.3:0/2240558375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 0x7f87dc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 --2- 172.21.15.3:0/2240558375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc103760 0x7f87dc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 --2- 172.21.15.3:0/2240558375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc102db0 0x7f87dc103190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.237+0000 7f87e20c3640 1 -- 172.21.15.3:0/2240558375 >> 172.21.15.3:0/2240558375 conn(0x7f87dc0fc7d0 msgr2=0x7f87dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.238+0000 7f87e20c3640 1 -- 172.21.15.3:0/2240558375 shutdown_connections 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.238+0000 7f87e20c3640 1 -- 172.21.15.3:0/2240558375 wait complete. 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.239+0000 7f87e20c3640 1 Processor -- start 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.239+0000 7f87e20c3640 1 -- start start 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 0x7f87dc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 0x7f87dc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc1120b0 con 0x7f87dc103760 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc111f30 con 0x7f87dc102db0 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87e20c3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc112230 con 0x7f87dc1082e0 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45722/0 (socket says 172.21.15.3:45722) 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 -- 172.21.15.3:0/3375498801 learned_addr learned my addr 172.21.15.3:0/3375498801 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87dbfff640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 0x7f87dc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 -- 172.21.15.3:0/3375498801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 msgr2=0x7f87dc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 0x7f87dc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87daffd640 1 -- 172.21.15.3:0/3375498801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 msgr2=0x7f87dc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.240+0000 7f87db7fe640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 0x7f87dc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87daffd640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 0x7f87dc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87daffd640 1 -- 172.21.15.3:0/3375498801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc19d520 con 0x7f87dc103760 2024-09-12T02:19:36.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87db7fe640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 0x7f87dc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87daffd640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f87d000d740 tx=0x7f87d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87d8ff9640 1 -- 172.21.15.3:0/3375498801 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0017070 con 0x7f87dc103760 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87d8ff9640 1 -- 172.21.15.3:0/3375498801 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d000de50 con 0x7f87dc103760 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.241+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc19d810 con 0x7f87dc103760 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.242+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc1a50f0 con 0x7f87dc103760 2024-09-12T02:19:36.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.242+0000 7f87d8ff9640 1 -- 172.21.15.3:0/3375498801 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d000f6b0 con 0x7f87dc103760 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.244+0000 7f87d8ff9640 1 -- 172.21.15.3:0/3375498801 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f87d001a070 con 0x7f87dc103760 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 Processor -- start 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 -- start start 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87d8ff9640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 0x7f87c40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069eb6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565069e0a620 con 0x565069ea5820 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565069db1e50 con 0x565069eb4650 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87e449c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565069e065e0 con 0x565069ea4f00 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.245+0000 7f87d8ff9640 1 -- 172.21.15.3:0/3375498801 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f87d00cee70 con 0x7f87dc103760 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87dbfff640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 0x7f87c40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45738/0 (socket says 172.21.15.3:45738) 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 -- 172.21.15.3:0/1060934594 learned_addr learned my addr 172.21.15.3:0/1060934594 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87dbfff640 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069eb6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 -- 172.21.15.3:0/1060934594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 msgr2=0x565069ea52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 -- 172.21.15.3:0/1060934594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 msgr2=0x565069eb6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069eb6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 -- 172.21.15.3:0/1060934594 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565069da2d40 con 0x565069ea5820 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87dbfff640 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069eb6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87dbfff640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 0x7f87c40a51d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d4009db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.246+0000 7f87daffd640 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f87d009b200 tx=0x7f87d009f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b3ef4d65b5c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.247+0000 7f87c1ffb640 1 -- 172.21.15.3:0/1060934594 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0021050 con 0x565069ea5820 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.247+0000 7f87c1ffb640 1 -- 172.21.15.3:0/1060934594 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d00cca00 con 0x565069ea5820 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.247+0000 7f87c1ffb640 1 -- 172.21.15.3:0/1060934594 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d00a3f40 con 0x565069ea5820 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- 172.21.15.3:0/1060934594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 msgr2=0x565069eb4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f87d009b200 tx=0x7f87d009f8b0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- 172.21.15.3:0/1060934594 shutdown_connections 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069eb6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069eb4110 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 --2- 172.21.15.3:0/1060934594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- 172.21.15.3:0/1060934594 >> 172.21.15.3:0/1060934594 conn(0x565069ea15f0 msgr2=0x565069ea19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- 172.21.15.3:0/1060934594 shutdown_connections 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- 172.21.15.3:0/1060934594 wait complete. 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 -- start start 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.248+0000 7f87e449c740 1 Processor -- start 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea5820 0x565069ead220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ead760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565069eb71c0 con 0x565069eb4650 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565069d92f10 con 0x565069ea5820 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565069eb7340 con 0x565069ea4f00 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87e449c740 10 client.0 _ll_register_callbacks cb 0x565069db6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea5820 0x565069ead220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:55002/0 (socket says 172.21.15.3:55002) 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 -- 172.21.15.3:0/367592388 learned_addr learned my addr 172.21.15.3:0/367592388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ead760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 -- 172.21.15.3:0/367592388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea5820 msgr2=0x565069ead220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea5820 0x565069ead220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 -- 172.21.15.3:0/367592388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 msgr2=0x565069ead760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ead760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.249+0000 7f87db7fe640 1 -- 172.21.15.3:0/367592388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565069b19580 con 0x565069ea4f00 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87dbfff640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87db7fe640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87cc033000 tx=0x7f87cc0044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac145e942a62619 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87baffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc035040 con 0x565069ea4f00 2024-09-12T02:19:36.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f87cc005450 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d00cca00 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87cc0046c0 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87e449c740 10 client.25474 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.250+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565069be3960 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc004d40 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87cc031540 con 0x565069ea4f00 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 10 client.25474 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 10 client.25474 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 20 client.25474 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069ec8720 0x565069ecab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 4 client.25474 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87d00cca00 con 0x565069ec8720 2024-09-12T02:19:36.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.251+0000 7f87daffd640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069ec8720 0x565069ecab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.252+0000 7f87daffd640 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069ec8720 0x565069ecab60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d850a3f7cb5d190 server_cookie=1ba99f214b768236 in_seq=0 out_seq=0 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.252+0000 7f87baffd640 10 client.25474 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87baffd640 1 -- 172.21.15.3:0/367592388 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f87d00cca00 con 0x565069ec8720 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87baffd640 10 client.25474 handle_command_reply: tid=0 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 client.25474 shutdown 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 msgr2=0x565069eb3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f87cc033000 tx=0x7f87cc0044f0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 shutdown_connections 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069ec8720 0x565069ecab60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87baffd640 0 client.25474 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea5820 0x565069ead220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 --2- 172.21.15.3:0/367592388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069eb3f20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.257 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 >> 172.21.15.3:0/367592388 conn(0x565069ea15f0 msgr2=0x565069ea7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 shutdown_connections 2024-09-12T02:19:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 1 -- 172.21.15.3:0/367592388 wait complete. 2024-09-12T02:19:36.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.253+0000 7f87e449c740 20 client.25474 trim_cache size 0 max 16384 2024-09-12T02:19:36.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.287+0000 7f87e449c740 1 Processor -- start 2024-09-12T02:19:36.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.287+0000 7f87e449c740 1 -- start start 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069ed0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800e470 con 0x565069ea5820 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45750/0 (socket says 172.21.15.3:45750) 2024-09-12T02:19:36.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87daffd640 1 -- 172.21.15.3:0/2705767170 learned_addr learned my addr 172.21.15.3:0/2705767170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800ffb0 con 0x565069eb4650 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.288+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800eb40 con 0x565069ea4f00 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87dbfff640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069ed0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 -- 172.21.15.3:0/2705767170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 msgr2=0x565069ed0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069ed0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 -- 172.21.15.3:0/2705767170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 msgr2=0x565069ed0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87db7fe640 1 -- 172.21.15.3:0/2705767170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565069eb7390 con 0x565069ea4f00 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87dbfff640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069ed0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.289+0000 7f87daffd640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.290+0000 7f87db7fe640 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea7ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02a940 tx=0x7f87cc02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396e29cd793cfc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.290+0000 7f87c17fa640 1 -- 172.21.15.3:0/2705767170 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc035070 con 0x565069ea4f00 2024-09-12T02:19:36.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.290+0000 7f87c17fa640 1 -- 172.21.15.3:0/2705767170 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc031540 con 0x565069ea4f00 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.290+0000 7f87c17fa640 1 -- 172.21.15.3:0/2705767170 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0035f0 con 0x565069ea4f00 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 msgr2=0x565069ea7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea7ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87cc02a940 tx=0x7f87cc02a420 comp rx=0 tx=0).stop 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 shutdown_connections 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069eb4650 0x565069ed0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069ea5820 0x565069ed0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 --2- 172.21.15.3:0/2705767170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea4f00 0x565069ea7ab0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.292 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 >> 172.21.15.3:0/2705767170 conn(0x565069ea15f0 msgr2=0x565069ea0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.291+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 shutdown_connections 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.292+0000 7f87e449c740 1 -- 172.21.15.3:0/2705767170 wait complete. 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.292+0000 7f87e449c740 1 -- start start 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.292+0000 7f87e449c740 1 Processor -- start 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea4f00 0x565069ec5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 0x565069ec19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800f8e0 con 0x565069eb4650 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800f210 con 0x565069ea4f00 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f87c800c450 con 0x565069ea5820 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87e449c740 10 client.0 _ll_register_callbacks cb 0x7f87cc004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45754/0 (socket says 172.21.15.3:45754) 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 -- 172.21.15.3:0/1586907338 learned_addr learned my addr 172.21.15.3:0/1586907338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87db7fe640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea4f00 0x565069ec5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87daffd640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 0x565069ec19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 -- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 msgr2=0x565069ec19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 0x565069ec19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 -- 172.21.15.3:0/1586907338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea4f00 msgr2=0x565069ec5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea4f00 0x565069ec5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87dbfff640 1 -- 172.21.15.3:0/1586907338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565069ec25e0 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.293+0000 7f87daffd640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 0x565069ec19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87dbfff640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f87d4010f30 tx=0x7f87d4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573a86d129e98d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87baffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4004740 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f87d401b040 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87cc031540 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f87d4004b20 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87e449c740 10 client.15849 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.294+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565069f34330 con 0x565069eb4650 2024-09-12T02:19:36.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc031540 con 0x565069eb4650 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f87d401f050 con 0x565069eb4650 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 10 client.15849 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 10 client.15849 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 20 client.15849 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069f34ab0 0x565069f36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 4 client.15849 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ddfbd753", "release": true, "await": true}] 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ddfbd753", "release": true, "await": true}) -- 0x565069e17b90 con 0x565069f34ab0 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.295+0000 7f87db7fe640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069f34ab0 0x565069f36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:36.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.296+0000 7f87db7fe640 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069f34ab0 0x565069f36ed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b14aec8496d3804 server_cookie=bfe7a5020268cc43 in_seq=0 out_seq=0 2024-09-12T02:19:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.296+0000 7f87baffd640 10 client.15849 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.298+0000 7f87baffd640 1 -- 172.21.15.3:0/1586907338 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x565069e17b90 con 0x565069f34ab0 2024-09-12T02:19:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.298+0000 7f87baffd640 10 client.15849 handle_command_reply: tid=0 2024-09-12T02:19:36.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.298+0000 7f87e449c740 1 client.15849 shutdown 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.298+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 msgr2=0x565069ec1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.298+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f87d4010f30 tx=0x7f87d4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 shutdown_connections 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565069f34ab0 0x565069f36ed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565069eb4650 0x565069ec1ef0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565069ea5820 0x565069ec19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 --2- 172.21.15.3:0/1586907338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565069ea4f00 0x565069ec5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 >> 172.21.15.3:0/1586907338 conn(0x565069ea15f0 msgr2=0x565069eb2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87baffd640 0 client.15849 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 shutdown_connections 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 1 -- 172.21.15.3:0/1586907338 wait complete. 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.299+0000 7f87e449c740 20 client.15849 trim_cache size 0 max 16384 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 29, 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 4, 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:19:36.300 INFO:teuthology.orchestra.run.smithi003.stdout: "ddfbd753": { 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 4, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.5, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:19:36.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:36.302 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 msgr2=0x7f87c40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 0x7f87c40a51d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005fd0 tx=0x7f87d4009db0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 msgr2=0x7f87dc198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f87d000d740 tx=0x7f87d000dc00 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 shutdown_connections 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f87c40a2d10 0x7f87c40a51d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87dc1082e0 0x7f87dc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f87dc103760 0x7f87dc198ab0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 --2- 172.21.15.3:0/3375498801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc102db0 0x7f87dc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 >> 172.21.15.3:0/3375498801 conn(0x7f87dc0fc7d0 msgr2=0x7f87dc106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 shutdown_connections 2024-09-12T02:19:36.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:36.300+0000 7f87e20c3640 1 -- 172.21.15.3:0/3375498801 wait complete. 2024-09-12T02:19:36.307 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: pgmap v372: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: scrub queued for path: / 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:36 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:19:36.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: pgmap v372: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: scrub queued for path: / 2024-09-12T02:19:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:36 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: pgmap v372: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: scrub queued for path: / 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:36 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:19:37.128 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ddfbd753', total seconds elapsed: 30.3 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 -- 172.21.15.3:0/1975946050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 msgr2=0x7fd6281029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1975946050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 0x7fd6281029b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009900 tx=0x7fd61c02a480 comp rx=0 tx=0).stop 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 -- 172.21.15.3:0/1975946050 shutdown_connections 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1975946050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 0x7fd62810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1975946050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 0x7fd6281029b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1975946050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd628101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.676+0000 7fd62df4d640 1 -- 172.21.15.3:0/1975946050 >> 172.21.15.3:0/1975946050 conn(0x7fd6280fc7d0 msgr2=0x7fd6280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.677+0000 7fd62df4d640 1 -- 172.21.15.3:0/1975946050 shutdown_connections 2024-09-12T02:19:37.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.677+0000 7fd62df4d640 1 -- 172.21.15.3:0/1975946050 wait complete. 2024-09-12T02:19:37.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.678+0000 7fd62df4d640 1 Processor -- start 2024-09-12T02:19:37.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.678+0000 7fd62df4d640 1 -- start start 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.678+0000 7fd62df4d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.678+0000 7fd62df4d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 0x7fd6281a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd62df4d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 0x7fd62810cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd62df4d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281120a0 con 0x7fd628102570 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd62df4d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd628111f20 con 0x7fd628100e20 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd62df4d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd628112220 con 0x7fd628102ef0 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40228/0 (socket says 172.21.15.3:40228) 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 -- 172.21.15.3:0/1038173631 learned_addr learned my addr 172.21.15.3:0/1038173631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:37.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd627fff640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 0x7fd62810cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 -- 172.21.15.3:0/1038173631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 msgr2=0x7fd62810cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 0x7fd62810cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 -- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 msgr2=0x7fd6281a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 0x7fd6281a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 -- 172.21.15.3:0/1038173631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62810d2b0 con 0x7fd628100e20 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.679+0000 7fd6277fe640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.680+0000 7fd624ff9640 1 -- 172.21.15.3:0/1038173631 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cd80 con 0x7fd628100e20 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.680+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62810d5a0 con 0x7fd628100e20 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.680+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281ab650 con 0x7fd628100e20 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.680+0000 7fd624ff9640 1 -- 172.21.15.3:0/1038173631 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd618005500 con 0x7fd628100e20 2024-09-12T02:19:37.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.680+0000 7fd624ff9640 1 -- 172.21.15.3:0/1038173631 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618010690 con 0x7fd628100e20 2024-09-12T02:19:37.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd624ff9640 1 -- 172.21.15.3:0/1038173631 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd6180046c0 con 0x7fd628100e20 2024-09-12T02:19:37.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 Processor -- start 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 -- start start 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46947880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b46952140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b468ad2e0 con 0x562b46947dc0 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b46854ce0 con 0x562b46952680 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.684+0000 7fd630326740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b468a92a0 con 0x562b469474a0 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40236/0 (socket says 172.21.15.3:40236) 2024-09-12T02:19:37.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 -- 172.21.15.3:0/108484973 learned_addr learned my addr 172.21.15.3:0/108484973 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:37.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd6277fe640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b46952140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd627fff640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46947880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 -- 172.21.15.3:0/108484973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 msgr2=0x562b46947880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46947880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 -- 172.21.15.3:0/108484973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 msgr2=0x562b46952140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b46952140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd626ffd640 1 -- 172.21.15.3:0/108484973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b46839440 con 0x562b46952680 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.685+0000 7fd6277fe640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b46952140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.686+0000 7fd626ffd640 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0058d0 tx=0x7fd61c005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b93149d3c2906f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.686+0000 7fd60dffb640 1 -- 172.21.15.3:0/108484973 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c03e030 con 0x562b46952680 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.686+0000 7fd60dffb640 1 -- 172.21.15.3:0/108484973 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c0035f0 con 0x562b46952680 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.686+0000 7fd60dffb640 1 -- 172.21.15.3:0/108484973 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c035040 con 0x562b46952680 2024-09-12T02:19:37.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 -- 172.21.15.3:0/108484973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 msgr2=0x562b46954a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0058d0 tx=0x7fd61c005900 comp rx=0 tx=0).stop 2024-09-12T02:19:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 -- 172.21.15.3:0/108484973 shutdown_connections 2024-09-12T02:19:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46952680 0x562b46954a70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b46952140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 --2- 172.21.15.3:0/108484973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46947880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 -- 172.21.15.3:0/108484973 >> 172.21.15.3:0/108484973 conn(0x562b46943a30 msgr2=0x562b46943e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.687+0000 7fd630326740 1 -- 172.21.15.3:0/108484973 shutdown_connections 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.688+0000 7fd624ff9640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 0x7fd6100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.688+0000 7fd630326740 1 -- 172.21.15.3:0/108484973 wait complete. 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.688+0000 7fd626ffd640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 0x7fd6100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.688+0000 7fd630326740 1 -- start start 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.688+0000 7fd624ff9640 1 -- 172.21.15.3:0/1038173631 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd6180cf500 con 0x7fd628100e20 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd630326740 1 Processor -- start 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd626ffd640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 0x7fd6100a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0096c0 tx=0x7fd61c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd630326740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46951ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd630326740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46947dc0 0x562b4694f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd630326740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46951ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd627fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46951ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:55062/0 (socket says 172.21.15.3:55062) 2024-09-12T02:19:37.691 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd627fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45794/0 (socket says 172.21.15.3:45794) 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd6277fe640 1 -- 172.21.15.3:0/903943008 learned_addr learned my addr 172.21.15.3:0/903943008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd626ffd640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46947dc0 0x562b4694f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.689+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b46966260 con 0x562b46952680 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562b46868050 con 0x562b46947dc0 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562b469663e0 con 0x562b469474a0 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 -- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 msgr2=0x562b46951ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46951ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 -- 172.21.15.3:0/903943008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46947dc0 msgr2=0x562b4694f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46947dc0 0x562b4694f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4661e3c0 con 0x562b46952680 2024-09-12T02:19:37.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.690+0000 7fd627fff640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd6200042c0 tx=0x7fd6200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbadf8f04a8640 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd630326740 10 client.0 _ll_register_callbacks cb 0x562b468a8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c0035f0 con 0x562b46952680 2024-09-12T02:19:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd606ffd640 10 client.15858 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:37.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd606ffd640 1 -- 172.21.15.3:0/903943008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004b20 con 0x562b46952680 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd606ffd640 1 -- 172.21.15.3:0/903943008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd620004cc0 con 0x562b46952680 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd606ffd640 1 -- 172.21.15.3:0/903943008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd620012a30 con 0x562b46952680 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd630326740 10 client.15858 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.691+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=29}) -- 0x562b46971af0 con 0x562b46952680 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd606ffd640 1 -- 172.21.15.3:0/903943008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd620012bd0 con 0x562b46952680 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 10 client.15858 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 20 client.15858 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b46972ea0 0x562b469752b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 4 client.15858 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd61c0035f0 con 0x562b46972ea0 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.692+0000 7fd6277fe640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b46972ea0 0x562b469752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.693+0000 7fd6277fe640 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b46972ea0 0x562b469752b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368c1436f2d6b826 server_cookie=ad0aae15fc2e530c in_seq=0 out_seq=0 2024-09-12T02:19:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.693+0000 7fd606ffd640 10 client.15858 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:37.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.693+0000 7fd606ffd640 1 -- 172.21.15.3:0/903943008 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd61c0035f0 con 0x562b46972ea0 2024-09-12T02:19:37.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.693+0000 7fd606ffd640 10 client.15858 handle_command_reply: tid=0 2024-09-12T02:19:37.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.693+0000 7fd630326740 1 client.15858 shutdown 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 msgr2=0x562b4694f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd6200042c0 tx=0x7fd6200046c0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 shutdown_connections 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b46972ea0 0x562b469752b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46952680 0x562b4694f6d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b46947dc0 0x562b4694f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 --2- 172.21.15.3:0/903943008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b469474a0 0x562b46951ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 >> 172.21.15.3:0/903943008 conn(0x562b46943a30 msgr2=0x562b46949760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd606ffd640 0 client.15858 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 shutdown_connections 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 1 -- 172.21.15.3:0/903943008 wait complete. 2024-09-12T02:19:37.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.694+0000 7fd630326740 20 client.15858 trim_cache size 0 max 16384 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.728+0000 7fd630326740 1 Processor -- start 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.728+0000 7fd630326740 1 -- start start 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4697abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b4694c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd61400e530 con 0x562b46947dc0 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd614010070 con 0x562b469474a0 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd630326740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd61400ec00 con 0x562b46952680 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45808/0 (socket says 172.21.15.3:45808) 2024-09-12T02:19:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.729+0000 7fd626ffd640 1 -- 172.21.15.3:0/4091785894 learned_addr learned my addr 172.21.15.3:0/4091785894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:37.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 -- 172.21.15.3:0/4091785894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 msgr2=0x562b4694c410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:19:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b4694c410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 -- 172.21.15.3:0/4091785894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 msgr2=0x562b4697abc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4697abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 -- 172.21.15.3:0/4091785894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b46947bf0 con 0x562b46947dc0 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd626ffd640 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x562b46944db0 tx=0x7fd61c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13c3013c80388e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd60d7fa640 1 -- 172.21.15.3:0/4091785894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c03e040 con 0x562b46947dc0 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd60d7fa640 1 -- 172.21.15.3:0/4091785894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c031a20 con 0x562b46947dc0 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.730+0000 7fd60d7fa640 1 -- 172.21.15.3:0/4091785894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0509a0 con 0x562b46947dc0 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.731+0000 7fd630326740 1 -- 172.21.15.3:0/4091785894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 msgr2=0x562b4697b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.731+0000 7fd630326740 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x562b46944db0 tx=0x7fd61c005900 comp rx=0 tx=0).stop 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 -- 172.21.15.3:0/4091785894 shutdown_connections 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b4694c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4697b100 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 --2- 172.21.15.3:0/4091785894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4697abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 -- 172.21.15.3:0/4091785894 >> 172.21.15.3:0/4091785894 conn(0x562b46943a30 msgr2=0x562b46942b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 -- 172.21.15.3:0/4091785894 shutdown_connections 2024-09-12T02:19:37.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 -- 172.21.15.3:0/4091785894 wait complete. 2024-09-12T02:19:37.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.732+0000 7fd630326740 1 -- start start 2024-09-12T02:19:37.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 Processor -- start 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4696b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b46969cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd61400f9a0 con 0x562b46947dc0 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd61400f2d0 con 0x562b469474a0 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd61400c510 con 0x562b46952680 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd630326740 10 client.0 _ll_register_callbacks cb 0x7fd620009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd6277fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4696b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd626ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45812/0 (socket says 172.21.15.3:45812) 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd626ffd640 1 -- 172.21.15.3:0/3220634539 learned_addr learned my addr 172.21.15.3:0/3220634539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.733+0000 7fd627fff640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b46969cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 -- 172.21.15.3:0/3220634539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 msgr2=0x562b4696b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4696b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 -- 172.21.15.3:0/3220634539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 msgr2=0x562b4696bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 -- 172.21.15.3:0/3220634539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4696a3a0 con 0x562b46952680 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd626ffd640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd627fff640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b46969cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd620000f80 tx=0x7fd620004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853807d02e340651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.734+0000 7fd606ffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620034060 con 0x562b46952680 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd62002c070 con 0x562b46952680 2024-09-12T02:19:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c031a20 con 0x562b46952680 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd620004d30 con 0x562b46952680 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd630326740 10 client.25492 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b469de750 con 0x562b46952680 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.735+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c031a20 con 0x562b46952680 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd62002f4e0 con 0x562b46952680 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 10 client.25492 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 20 client.25492 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b469deed0 0x562b469e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 4 client.25492 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b468ba850 con 0x562b469deed0 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.736+0000 7fd6277fe640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b469deed0 0x562b469e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd606ffd640 10 client.25492 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd6277fe640 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b469deed0 0x562b469e12f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe90bf2b27ded3f server_cookie=525d5d8c95dd4b32 in_seq=0 out_seq=0 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd606ffd640 1 -- 172.21.15.3:0/3220634539 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562b468ba850 con 0x562b469deed0 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd606ffd640 10 client.25492 handle_command_reply: tid=0 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 client.25492 shutdown 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 msgr2=0x562b46969cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b46969cb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd620000f80 tx=0x7fd620004a40 comp rx=0 tx=0).stop 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 shutdown_connections 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562b469deed0 0x562b469e12f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.741 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b46952680 0x562b46969cb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b46947dc0 0x562b4696bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 --2- 172.21.15.3:0/3220634539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562b469474a0 0x562b4696b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 >> 172.21.15.3:0/3220634539 conn(0x562b46943a30 msgr2=0x562b469509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.737+0000 7fd606ffd640 0 client.25492 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 shutdown_connections 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd630326740 1 -- 172.21.15.3:0/3220634539 wait complete. 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd630326740 20 client.25492 trim_cache size 0 max 16384 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 msgr2=0x7fd6100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 0x7fd6100a5050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0096c0 tx=0x7fd61c000bc0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 msgr2=0x7fd6281a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.738+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800edf0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 shutdown_connections 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd6100a2b90 0x7fd6100a5050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd628102ef0 0x7fd62810cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd628102570 0x7fd6281a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 --2- 172.21.15.3:0/1038173631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd628100e20 0x7fd6281a4780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 >> 172.21.15.3:0/1038173631 conn(0x7fd6280fc7d0 msgr2=0x7fd628106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 shutdown_connections 2024-09-12T02:19:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:37.739+0000 7fd62df4d640 1 -- 172.21.15.3:0/1038173631 wait complete. 2024-09-12T02:19:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:38 smithi003 ceph-mon[31476]: pgmap v373: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T02:19:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:38 smithi003 ceph-mon[31476]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:19:38.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:38 smithi092 ceph-mon[36089]: pgmap v373: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T02:19:38.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:38 smithi092 ceph-mon[36089]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:19:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:38 smithi026 ceph-mon[35496]: pgmap v373: 97 pgs: 97 active+clean; 670 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-12T02:19:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:38 smithi026 ceph-mon[35496]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:19:39.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:39 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:19:39.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:39 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:19:39.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:39 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:19:40.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:40 smithi026 ceph-mon[35496]: pgmap v374: 97 pgs: 97 active+clean; 671 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T02:19:40.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:40 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:40 smithi003 ceph-mon[31476]: pgmap v374: 97 pgs: 97 active+clean; 671 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T02:19:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:40 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:40 smithi092 ceph-mon[36089]: pgmap v374: 97 pgs: 97 active+clean; 671 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-12T02:19:40.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:40 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:40.885 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:19:40.885 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:19:41.258 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:41.615 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:19:41.615 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:19:41.615 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 -- 172.21.15.3:0/3944275244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d81008c0 msgr2=0x7fe6d8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/3944275244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d81008c0 0x7fe6d8100ca0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802a410 comp rx=0 tx=0).stop 2024-09-12T02:19:41.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 -- 172.21.15.3:0/3944275244 shutdown_connections 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/3944275244 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d8105ec0 0x7fe6d8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/3944275244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/3944275244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d81008c0 0x7fe6d8100ca0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 -- 172.21.15.3:0/3944275244 >> 172.21.15.3:0/3944275244 conn(0x7fe6d80fc7d0 msgr2=0x7fe6d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.710+0000 7fe6ddee8640 1 -- 172.21.15.3:0/3944275244 shutdown_connections 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 -- 172.21.15.3:0/3944275244 wait complete. 2024-09-12T02:19:41.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 Processor -- start 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 -- start start 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d81008c0 0x7fe6d81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 0x7fe6d819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8118630 con 0x7fe6d8105ec0 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.711+0000 7fe6ddee8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81184b0 con 0x7fe6d81011e0 2024-09-12T02:19:41.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6ddee8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81187b0 con 0x7fe6d81008c0 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 0x7fe6d819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38030/0 (socket says 172.21.15.3:38030) 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6cffff640 1 -- 172.21.15.3:0/599740744 learned_addr learned my addr 172.21.15.3:0/599740744 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d77fe640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d81008c0 0x7fe6d81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 -- 172.21.15.3:0/599740744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d81008c0 msgr2=0x7fe6d81a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:41.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d81008c0 0x7fe6d81a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 -- 172.21.15.3:0/599740744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 msgr2=0x7fe6d81a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 -- 172.21.15.3:0/599740744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d819f070 con 0x7fe6d8105ec0 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6cffff640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d7fff640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 0x7fe6d819e940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000e940 tx=0x7fe6d000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:41.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004d20 con 0x7fe6d8105ec0 2024-09-12T02:19:41.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0005020 con 0x7fe6d8105ec0 2024-09-12T02:19:41.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d819f360 con 0x7fe6d8105ec0 2024-09-12T02:19:41.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00116b0 con 0x7fe6d8105ec0 2024-09-12T02:19:41.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.712+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81ab650 con 0x7fe6d8105ec0 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.714+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d806a850 con 0x7fe6d8105ec0 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.715+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe6d00054d0 con 0x7fe6d8105ec0 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.715+0000 7fe6d57fa640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 0x7fe6c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.716+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe6d00cf390 con 0x7fe6d8105ec0 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.716+0000 7fe6d77fe640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 0x7fe6c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:41.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.716+0000 7fe6d77fe640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 0x7fe6c40a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6d819fa80 tx=0x7fe6c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:41.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.720+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6d0016030 con 0x7fe6d8105ec0 2024-09-12T02:19:41.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:19:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:19:41.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:41.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:19:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:19:41.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:19:41.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.963+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d81ab880 con 0x7fe6d8105ec0 2024-09-12T02:19:41.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.964+0000 7fe6d57fa640 1 -- 172.21.15.3:0/599740744 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11946 (secure 0 0 0) 0x7fe6d009bc90 con 0x7fe6d8105ec0 2024-09-12T02:19:41.966 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:19:41.966 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":29,"btime":"2024-09-12T02:18:21:329028+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:18:21.173770+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:19:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 29 2024-09-12T02:19:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 msgr2=0x7fe6c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 0x7fe6c40a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe6d819fa80 tx=0x7fe6c80058d0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 msgr2=0x7fe6d819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 0x7fe6d819e940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000e940 tx=0x7fe6d000ee00 comp rx=0 tx=0).stop 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 shutdown_connections 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe6c40a2b80 0x7fe6c40a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe6d8105ec0 0x7fe6d819e940 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6d81011e0 0x7fe6d81a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 --2- 172.21.15.3:0/599740744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6d81008c0 0x7fe6d81a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.966+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 >> 172.21.15.3:0/599740744 conn(0x7fe6d80fc7d0 msgr2=0x7fe6d8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.967+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 shutdown_connections 2024-09-12T02:19:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:41.967+0000 7fe6ddee8640 1 -- 172.21.15.3:0/599740744 wait complete. 2024-09-12T02:19:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:42 smithi026 ceph-mon[35496]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:42 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/599740744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:42 smithi003 ceph-mon[31476]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:42 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/599740744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:42.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:42 smithi092 ceph-mon[36089]: pgmap v375: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:42 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/599740744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:19:42.944 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:19:43.207 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 -- 172.21.15.3:0/2833592428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 msgr2=0x7f8c80101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 --2- 172.21.15.3:0/2833592428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c80101200 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009900 tx=0x7f8c7c02a480 comp rx=0 tx=0).stop 2024-09-12T02:19:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 -- 172.21.15.3:0/2833592428 shutdown_connections 2024-09-12T02:19:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 --2- 172.21.15.3:0/2833592428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 --2- 172.21.15.3:0/2833592428 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 --2- 172.21.15.3:0/2833592428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c80101200 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 -- 172.21.15.3:0/2833592428 >> 172.21.15.3:0/2833592428 conn(0x7f8c800fc7d0 msgr2=0x7f8c800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 -- 172.21.15.3:0/2833592428 shutdown_connections 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.751+0000 7f8c874c8640 1 -- 172.21.15.3:0/2833592428 wait complete. 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 Processor -- start 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 -- start start 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c801a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80110fc0 con 0x7f8c80100e20 2024-09-12T02:19:43.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80110e40 con 0x7f8c80102ef0 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c874c8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c80111140 con 0x7f8c80102570 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c85264640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c85264640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33338/0 (socket says 172.21.15.3:33338) 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c85264640 1 -- 172.21.15.3:0/513483180 learned_addr learned my addr 172.21.15.3:0/513483180 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c84a63640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.752+0000 7f8c85a65640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c801a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 -- 172.21.15.3:0/513483180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 msgr2=0x7f8c801998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 -- 172.21.15.3:0/513483180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 msgr2=0x7f8c801a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c801a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 -- 172.21.15.3:0/513483180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c801a4960 con 0x7f8c80100e20 2024-09-12T02:19:43.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c84a63640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85a65640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c801a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c85264640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006010 tx=0x7f8c7c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c767fc640 1 -- 172.21.15.3:0/513483180 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c03f070 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c767fc640 1 -- 172.21.15.3:0/513483180 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c031c30 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c801a4b30 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c767fc640 1 -- 172.21.15.3:0/513483180 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c035030 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.753+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801a4f80 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.755+0000 7f8c767fc640 1 -- 172.21.15.3:0/513483180 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8c7c042070 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.756+0000 7f8c767fc640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.756+0000 7f8c84a63640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.756+0000 7f8c767fc640 1 -- 172.21.15.3:0/513483180 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8c7c0ef960 con 0x7f8c80100e20 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c84a63640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c700058c0 tx=0x7f8c70005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 Processor -- start 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 -- start start 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfdc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559a4df422f0 con 0x559a4dfdc5d0 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559a4dee9cf0 con 0x559a4dfe7370 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c898a1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a4df3e2b0 con 0x559a4dfddb40 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.757+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33346/0 (socket says 172.21.15.3:33346) 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 -- 172.21.15.3:0/399946451 learned_addr learned my addr 172.21.15.3:0/399946451 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c85264640 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfdc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c85a65640 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 -- 172.21.15.3:0/399946451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 msgr2=0x559a4dfdc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfdc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 -- 172.21.15.3:0/399946451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 msgr2=0x559a4dfe9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 -- 172.21.15.3:0/399946451 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4dece3a0 con 0x559a4dfdc5d0 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c84a63640 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bfc0 tx=0x7f8c70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f34dcd432490500 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c6b7fe640 1 -- 172.21.15.3:0/399946451 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70022070 con 0x559a4dfdc5d0 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c6b7fe640 1 -- 172.21.15.3:0/399946451 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c70011e50 con 0x559a4dfdc5d0 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.758+0000 7f8c6b7fe640 1 -- 172.21.15.3:0/399946451 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70010b20 con 0x559a4dfdc5d0 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- 172.21.15.3:0/399946451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 msgr2=0x559a4dfdca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bfc0 tx=0x7f8c70012000 comp rx=0 tx=0).stop 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- 172.21.15.3:0/399946451 shutdown_connections 2024-09-12T02:19:43.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfdca10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 --2- 172.21.15.3:0/399946451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfdc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- 172.21.15.3:0/399946451 >> 172.21.15.3:0/399946451 conn(0x559a4dfa5310 msgr2=0x559a4dfd8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- 172.21.15.3:0/399946451 shutdown_connections 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- 172.21.15.3:0/399946451 wait complete. 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.759+0000 7f8c898a1740 1 -- start start 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 Processor -- start 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfe3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559a4dfea070 con 0x559a4dfdc5d0 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559a4defd060 con 0x559a4dfe7370 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559a4dfea1f0 con 0x559a4dfddb40 2024-09-12T02:19:43.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c898a1740 10 client.0 _ll_register_callbacks cb 0x559a4dc508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33362/0 (socket says 172.21.15.3:33362) 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 -- 172.21.15.3:0/758770793 learned_addr learned my addr 172.21.15.3:0/758770793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c84a63640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfe3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85a65640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 -- 172.21.15.3:0/758770793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 msgr2=0x559a4dfe3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfe3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 -- 172.21.15.3:0/758770793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 msgr2=0x559a4dfe45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.760+0000 7f8c85264640 1 -- 172.21.15.3:0/758770793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4dcb33c0 con 0x559a4dfdc5d0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c85264640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ed450 tx=0x7f8c7c0bbb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fce792139c6258 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:43.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0d5060 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8c7c0b8820 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c70011e50 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c049040 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 10 client.15882 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a4dff5b10 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c70011e50 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c7c0c0170 con 0x559a4dfdc5d0 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 10 client.15882 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 10 client.15882 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 20 client.15882 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4dff6f70 0x559a4dff93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 4 client.15882 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:43.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.761+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a4df4f860 con 0x559a4dff6f70 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.762+0000 7f8c84a63640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4dff6f70 0x559a4dff93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.762+0000 7f8c68ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.762+0000 7f8c84a63640 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4dff6f70 0x559a4dff93a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aded62ab401ec606 server_cookie=e21451b53fb2f75b in_seq=0 out_seq=0 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c68ff9640 1 -- 172.21.15.3:0/758770793 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a4df4f860 con 0x559a4dff6f70 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c68ff9640 10 client.15882 handle_command_reply: tid=0 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 client.15882 shutdown 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 msgr2=0x559a4dfe6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ed450 tx=0x7f8c7c0bbb10 comp rx=0 tx=0).stop 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 shutdown_connections 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfe7370 0x559a4dfe45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4dff6f70 0x559a4dff93a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfddb40 0x559a4dfe3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 --2- 172.21.15.3:0/758770793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe6d40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 >> 172.21.15.3:0/758770793 conn(0x559a4dfa5310 msgr2=0x559a4dfde930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c68ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 shutdown_connections 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 1 -- 172.21.15.3:0/758770793 wait complete. 2024-09-12T02:19:43.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.763+0000 7f8c898a1740 20 client.15882 trim_cache size 0 max 16384 2024-09-12T02:19:43.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 Processor -- start 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 -- start start 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfdf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801ad5d0 con 0x559a4dfddb40 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801af110 con 0x559a4dfdc5d0 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.790+0000 7f8c898a1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801adca0 con 0x559a4dfe7370 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33374/0 (socket says 172.21.15.3:33374) 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 -- 172.21.15.3:0/903544668 learned_addr learned my addr 172.21.15.3:0/903544668 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:43.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c85a65640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfdf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c85264640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 -- 172.21.15.3:0/903544668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 msgr2=0x559a4dfdf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfdf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 -- 172.21.15.3:0/903544668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 msgr2=0x559a4dfe2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 -- 172.21.15.3:0/903544668 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4dfea160 con 0x559a4dfddb40 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c85a65640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfdf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c84a63640 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x559a4dfd9ac0 tx=0x7f8c70015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ad5cb6728e9473 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c6affd640 1 -- 172.21.15.3:0/903544668 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70022040 con 0x559a4dfddb40 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c6affd640 1 -- 172.21.15.3:0/903544668 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c700106d0 con 0x559a4dfddb40 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.791+0000 7f8c6affd640 1 -- 172.21.15.3:0/903544668 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700048e0 con 0x559a4dfddb40 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 -- 172.21.15.3:0/903544668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 msgr2=0x559a4dfd7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x559a4dfd9ac0 tx=0x7f8c70015000 comp rx=0 tx=0).stop 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 -- 172.21.15.3:0/903544668 shutdown_connections 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfdf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfd7bf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 --2- 172.21.15.3:0/903544668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfe2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 -- 172.21.15.3:0/903544668 >> 172.21.15.3:0/903544668 conn(0x559a4dfa5310 msgr2=0x559a4dfd74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 -- 172.21.15.3:0/903544668 shutdown_connections 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.792+0000 7f8c898a1740 1 -- 172.21.15.3:0/903544668 wait complete. 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 -- start start 2024-09-12T02:19:43.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 Processor -- start 2024-09-12T02:19:43.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfeae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801aea40 con 0x559a4dfddb40 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801ae370 con 0x559a4dfdc5d0 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8c801ab5b0 con 0x559a4dfe7370 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c898a1740 10 client.0 _ll_register_callbacks cb 0x7f8c7c0b7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c84a63640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33390/0 (socket says 172.21.15.3:33390) 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.793+0000 7f8c84a63640 1 -- 172.21.15.3:0/962314043 learned_addr learned my addr 172.21.15.3:0/962314043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c85264640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c85a65640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfeae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 -- 172.21.15.3:0/962314043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 msgr2=0x559a4dfeae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfeae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 -- 172.21.15.3:0/962314043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 msgr2=0x559a4dfef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 -- 172.21.15.3:0/962314043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4dfeb630 con 0x559a4dfddb40 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c85264640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c84a63640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bf90 tx=0x7f8c7000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d4299fe2a3fef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70045060 con 0x559a4dfddb40 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8c70010c90 con 0x559a4dfddb40 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c700106d0 con 0x559a4dfddb40 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c70005030 con 0x559a4dfddb40 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c898a1740 10 client.15888 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.794+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a4e0618e0 con 0x559a4dfddb40 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c700106d0 con 0x559a4dfddb40 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c70049020 con 0x559a4dfddb40 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 10 client.15888 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 10 client.15888 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 20 client.15888 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4e061e10 0x559a4e064230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 4 client.15888 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a4df49d70 con 0x559a4e061e10 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c85264640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4e061e10 0x559a4e064230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.795+0000 7f8c68ff9640 10 client.15888 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c85264640 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4e061e10 0x559a4e064230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e701c09fac170bb2 server_cookie=1147bcbab45050c9 in_seq=0 out_seq=0 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c68ff9640 1 -- 172.21.15.3:0/962314043 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a4df49d70 con 0x559a4e061e10 2024-09-12T02:19:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c68ff9640 10 client.15888 handle_command_reply: tid=0 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 client.15888 shutdown 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 msgr2=0x559a4dfefd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000bf90 tx=0x7f8c7000bd30 comp rx=0 tx=0).stop 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 shutdown_connections 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x559a4e061e10 0x559a4e064230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559a4dfe7370 0x559a4dfeae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559a4dfddb40 0x559a4dfefd70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c68ff9640 0 client.15888 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 --2- 172.21.15.3:0/962314043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559a4dfdc5d0 0x559a4dfef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 >> 172.21.15.3:0/962314043 conn(0x559a4dfa5310 msgr2=0x559a4dfd7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.796+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 shutdown_connections 2024-09-12T02:19:43.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c898a1740 1 -- 172.21.15.3:0/962314043 wait complete. 2024-09-12T02:19:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c898a1740 20 client.15888 trim_cache size 0 max 16384 2024-09-12T02:19:43.803 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:43.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 msgr2=0x7f8c6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c700058c0 tx=0x7f8c70005850 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 msgr2=0x7f8c801993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006010 tx=0x7f8c7c003c10 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 shutdown_connections 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8c6c0a2b80 0x7f8c6c0a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c80102ef0 0x7f8c801a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c80102570 0x7f8c801998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 --2- 172.21.15.3:0/513483180 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c80100e20 0x7f8c801993a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.797+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 >> 172.21.15.3:0/513483180 conn(0x7f8c800fc7d0 msgr2=0x7f8c800fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.798+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 shutdown_connections 2024-09-12T02:19:43.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:43.798+0000 7f8c874c8640 1 -- 172.21.15.3:0/513483180 wait complete. 2024-09-12T02:19:44.398 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 86% (11177/12903) Updating files: 87% (11226/12903) Updating files: 88% (11355/12903) Updating files: 89% (11484/12903) Updating files: 90% (11613/12903) Updating files: 91% (11742/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T02:19:44.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:44 smithi003 ceph-mon[31476]: pgmap v376: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:44.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:19:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:44 smithi026 ceph-mon[35496]: pgmap v376: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:44 smithi092 ceph-mon[36089]: pgmap v376: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:44.995 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T02:19:45.116 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-09-12T02:19:45.117 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T02:19:45.121 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T02:19:45.182 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T02:19:45.185 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:19:45.185 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 -- 172.21.15.3:0/765183478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d01129f0 msgr2=0x7f85d0114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/765183478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d01129f0 0x7f85d0114de0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009ca0 tx=0x7f85c802a950 comp rx=0 tx=0).stop 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 -- 172.21.15.3:0/765183478 shutdown_connections 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/765183478 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d01129f0 0x7f85d0114de0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/765183478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d0074eb0 0x7f85d01124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/765183478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 -- 172.21.15.3:0/765183478 >> 172.21.15.3:0/765183478 conn(0x7f85d00fd700 msgr2=0x7f85d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 -- 172.21.15.3:0/765183478 shutdown_connections 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.579+0000 7f85d4f9f640 1 -- 172.21.15.3:0/765183478 wait complete. 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 Processor -- start 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 -- start start 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d01129f0 0x7f85d019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01184e0 con 0x7f85d0074eb0 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0118360 con 0x7f85d01129f0 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.580+0000 7f85d4f9f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0118660 con 0x7f85d0074590 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33406/0 (socket says 172.21.15.3:33406) 2024-09-12T02:19:45.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 -- 172.21.15.3:0/1453633406 learned_addr learned my addr 172.21.15.3:0/1453633406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85cffff640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d01129f0 0x7f85d019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 -- 172.21.15.3:0/1453633406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 msgr2=0x7f85d01a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d01a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 -- 172.21.15.3:0/1453633406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d01129f0 msgr2=0x7f85d019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d01129f0 0x7f85d019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 -- 172.21.15.3:0/1453633406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d019ef80 con 0x7f85d0074eb0 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c970 tx=0x7f85c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ccff9640 1 -- 172.21.15.3:0/1453633406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4007c10 con 0x7f85d0074eb0 2024-09-12T02:19:45.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ccff9640 1 -- 172.21.15.3:0/1453633406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4007db0 con 0x7f85d0074eb0 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d019f270 con 0x7f85d0074eb0 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85ccff9640 1 -- 172.21.15.3:0/1453633406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c400f940 con 0x7f85d0074eb0 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.581+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01ab5c0 con 0x7f85d0074eb0 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85ccff9640 1 -- 172.21.15.3:0/1453633406 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f85c400fae0 con 0x7f85d0074eb0 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 Processor -- start 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 -- start start 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85ccff9640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 0x7f85b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c103d20 0x55d58c102270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d58c0682f0 con 0x55d58c10d550 2024-09-12T02:19:45.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d58c00fcf0 con 0x55d58c1027b0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85d85da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d58c0642b0 con 0x55d58c103d20 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.584+0000 7f85cffff640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 0x7f85b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85ccff9640 1 -- 172.21.15.3:0/1453633406 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f85c40cf3c0 con 0x7f85d0074eb0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33408/0 (socket says 172.21.15.3:33408) 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 -- 172.21.15.3:0/3508030758 learned_addr learned my addr 172.21.15.3:0/3508030758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85ceffd640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c103d20 0x55d58c102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 -- 172.21.15.3:0/3508030758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c103d20 msgr2=0x55d58c102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c103d20 0x55d58c102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 -- 172.21.15.3:0/3508030758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 msgr2=0x55d58c102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 -- 172.21.15.3:0/3508030758 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58bff43a0 con 0x55d58c10d550 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85ceffd640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 0x7f85b80a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85c8030040 tx=0x7f85c8009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85cffff640 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f85d0105c50 tx=0x7f85c8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5424d9932e26fee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85b5ffb640 1 -- 172.21.15.3:0/3508030758 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8034040 con 0x55d58c10d550 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85b5ffb640 1 -- 172.21.15.3:0/3508030758 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8003d40 con 0x55d58c10d550 2024-09-12T02:19:45.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.585+0000 7f85b5ffb640 1 -- 172.21.15.3:0/3508030758 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c804a470 con 0x55d58c10d550 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- 172.21.15.3:0/3508030758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 msgr2=0x55d58c10f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f85d0105c50 tx=0x7f85c8048000 comp rx=0 tx=0).stop 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- 172.21.15.3:0/3508030758 shutdown_connections 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c10d550 0x55d58c10f940 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 --2- 172.21.15.3:0/3508030758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c103d20 0x55d58c102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- 172.21.15.3:0/3508030758 >> 172.21.15.3:0/3508030758 conn(0x55d58c0cb310 msgr2=0x55d58c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- 172.21.15.3:0/3508030758 shutdown_connections 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- 172.21.15.3:0/3508030758 wait complete. 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.586+0000 7f85d85da740 1 -- start start 2024-09-12T02:19:45.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 Processor -- start 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c10ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c10a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d58c1101f0 con 0x55d58c103d20 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d58c023060 con 0x55d58c1027b0 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d58c110370 con 0x55d58c10d550 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85d85da740 10 client.0 _ll_register_callbacks cb 0x55d58bd768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33418/0 (socket says 172.21.15.3:33418) 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 -- 172.21.15.3:0/169632918 learned_addr learned my addr 172.21.15.3:0/169632918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:45.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85cffff640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c10ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 -- 172.21.15.3:0/169632918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 msgr2=0x55d58c10a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c10a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 -- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 msgr2=0x55d58c10ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c10ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85ceffd640 1 -- 172.21.15.3:0/169632918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58bdd93c0 con 0x55d58c103d20 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.587+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c10ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85ceffd640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f85c409bbd0 tx=0x7f85c409c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751deda6ab44a601 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40a4820 con 0x55d58c103d20 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f85c40ccf60 con 0x55d58c103d20 2024-09-12T02:19:45.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c8003d40 con 0x55d58c103d20 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85c4018b80 con 0x55d58c103d20 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 10 client.15900 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d58c11bc70 con 0x55d58c103d20 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8003d40 con 0x55d58c103d20 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85c409b090 con 0x55d58c103d20 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 10 client.15900 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 10 client.15900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 20 client.15900 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c11d0d0 0x55d58c11f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 4 client.15900 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.588+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d58c075860 con 0x55d58c11d0d0 2024-09-12T02:19:45.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.589+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c11d0d0 0x55d58c11f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.589+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c11d0d0 0x55d58c11f4e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f710370336da8113 server_cookie=14bd6323e87a6b55 in_seq=0 out_seq=0 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.589+0000 7f85aeffd640 10 client.15900 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85aeffd640 1 -- 172.21.15.3:0/169632918 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d58c075860 con 0x55d58c11d0d0 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85aeffd640 10 client.15900 handle_command_reply: tid=0 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 client.15900 shutdown 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 msgr2=0x55d58c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f85c409bbd0 tx=0x7f85c409c400 comp rx=0 tx=0).stop 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 shutdown_connections 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c10a190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c11d0d0 0x55d58c11f4e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 --2- 172.21.15.3:0/169632918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c10ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85aeffd640 0 client.15900 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 >> 172.21.15.3:0/169632918 conn(0x55d58c0cb310 msgr2=0x55d58c104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 shutdown_connections 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 1 -- 172.21.15.3:0/169632918 wait complete. 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.590+0000 7f85d85da740 20 client.15900 trim_cache size 0 max 16384 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 Processor -- start 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 -- start start 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c103d20 0x55d58c124db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c1252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b3a20 con 0x55d58c1027b0 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b5560 con 0x55d58c103d20 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85d85da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b40f0 con 0x55d58c10d550 2024-09-12T02:19:45.656 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33430/0 (socket says 172.21.15.3:33430) 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85cf7fe640 1 -- 172.21.15.3:0/1894611102 learned_addr learned my addr 172.21.15.3:0/1894611102 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.603+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c103d20 0x55d58c124db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cffff640 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c1252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 -- 172.21.15.3:0/1894611102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 msgr2=0x55d58c1252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c1252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 -- 172.21.15.3:0/1894611102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c103d20 msgr2=0x55d58c124db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c103d20 0x55d58c124db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 -- 172.21.15.3:0/1894611102 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58c1086e0 con 0x55d58c1027b0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55d58c11fcf0 tx=0x7f85c00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccbe7342a4111de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85b57fa640 1 -- 172.21.15.3:0/1894611102 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c001f070 con 0x55d58c1027b0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85b57fa640 1 -- 172.21.15.3:0/1894611102 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c001ba60 con 0x55d58c1027b0 2024-09-12T02:19:45.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.604+0000 7f85b57fa640 1 -- 172.21.15.3:0/1894611102 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0011d70 con 0x55d58c1027b0 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- 172.21.15.3:0/1894611102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 msgr2=0x55d58c1194e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x55d58c11fcf0 tx=0x7f85c00094a0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- 172.21.15.3:0/1894611102 shutdown_connections 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c1252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c103d20 0x55d58c124db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- 172.21.15.3:0/1894611102 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c1027b0 0x55d58c1194e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- 172.21.15.3:0/1894611102 >> 172.21.15.3:0/1894611102 conn(0x55d58c0cb310 msgr2=0x55d58c109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- 172.21.15.3:0/1894611102 shutdown_connections 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- 172.21.15.3:0/1894611102 wait complete. 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- start start 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 Processor -- start 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c111820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b4e90 con 0x55d58c103d20 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b47c0 con 0x55d58c1027b0 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.605+0000 7f85d85da740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01b1a00 con 0x55d58c10d550 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85d85da740 10 client.0 _ll_register_callbacks cb 0x7f85c400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33440/0 (socket says 172.21.15.3:33440) 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 -- 172.21.15.3:0/1299106790 learned_addr learned my addr 172.21.15.3:0/1299106790 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85cffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c115eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 -- 172.21.15.3:0/1299106790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 msgr2=0x55d58c111820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c111820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 -- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 msgr2=0x55d58c115eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c115eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 -- 172.21.15.3:0/1299106790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58c112050 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85ceffd640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f85c4098050 tx=0x7f85c4018e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15196102162e769 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40a4820 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f85c40a4390 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c001ba60 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85c40a45d0 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85d85da740 10 client.15906 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.606+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d58c187e90 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c001ba60 con 0x55d58c103d20 2024-09-12T02:19:45.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85c407ab40 con 0x55d58c103d20 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 10 client.15906 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 10 client.15906 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 20 client.15906 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c188560 0x55d58c18a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 4 client.15906 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d58c06fd70 con 0x55d58c188560 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c188560 0x55d58c18a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85cf7fe640 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c188560 0x55d58c18a950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1fbbe8c1d46cf048 server_cookie=b1e8dd0c166fcd26 in_seq=0 out_seq=0 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.607+0000 7f85aeffd640 10 client.15906 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85aeffd640 1 -- 172.21.15.3:0/1299106790 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d58c06fd70 con 0x55d58c188560 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85aeffd640 10 client.15906 handle_command_reply: tid=0 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 client.15906 shutdown 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 msgr2=0x55d58c111230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f85c4098050 tx=0x7f85c4018e80 comp rx=0 tx=0).stop 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 shutdown_connections 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d58c10d550 0x55d58c111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d58c103d20 0x55d58c111230 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x55d58c188560 0x55d58c18a950 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 --2- 172.21.15.3:0/1299106790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d58c1027b0 0x55d58c115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 >> 172.21.15.3:0/1299106790 conn(0x55d58c0cb310 msgr2=0x55d58c10b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85aeffd640 0 client.15906 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:19:45.662 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 shutdown_connections 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 1 -- 172.21.15.3:0/1299106790 wait complete. 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d85da740 20 client.15906 trim_cache size 0 max 16384 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 msgr2=0x7f85b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 0x7f85b80a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f85c8030040 tx=0x7f85c8009890 comp rx=0 tx=0).stop 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 msgr2=0x7f85d01a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.608+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f85c400c970 tx=0x7f85c400ce30 comp rx=0 tx=0).stop 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 shutdown_connections 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f85b80a2c00 0x7f85b80a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85d01129f0 0x7f85d019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f85d0074eb0 0x7f85d01a4b80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 --2- 172.21.15.3:0/1453633406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85d0074590 0x7f85d01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 >> 172.21.15.3:0/1453633406 conn(0x7f85d00fd700 msgr2=0x7f85d00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 shutdown_connections 2024-09-12T02:19:45.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:45.609+0000 7f85d4f9f640 1 -- 172.21.15.3:0/1453633406 wait complete. 2024-09-12T02:19:46.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:46 smithi003 ceph-mon[31476]: pgmap v377: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:46.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:19:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:46 smithi026 ceph-mon[35496]: pgmap v377: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:46.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:46 smithi092 ceph-mon[36089]: pgmap v377: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:19:46.890 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:46.908 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T02:19:47.045 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T02:19:47.089 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:19:47.089 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:19:47.089 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T02:19:47.129 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:19:47.132 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:19:47.132 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T02:19:47.149 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-12T02:19:47.150 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-12T02:19:47.150 DEBUG:teuthology.orchestra.run.smithi003:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T02:19:47.214 INFO:tasks.workunit.client.0.smithi003.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-12T02:19:47.214 INFO:tasks.workunit.client.0.smithi003.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-12T02:19:47.214 INFO:tasks.workunit.client.0.smithi003.stderr:+ trap -- retry EXIT 2024-09-12T02:19:47.215 INFO:tasks.workunit.client.0.smithi003.stderr:+ rm -rf ffsb 2024-09-12T02:19:47.215 INFO:tasks.workunit.client.0.smithi003.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-12T02:19:47.255 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into 'ffsb'... 2024-09-12T02:19:47.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 -- 172.21.15.3:0/3869764552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc102790 msgr2=0x7f26fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 --2- 172.21.15.3:0/3869764552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc102790 0x7f26fc1067e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009900 tx=0x7f26ec02a410 comp rx=0 tx=0).stop 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 -- 172.21.15.3:0/3869764552 shutdown_connections 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 --2- 172.21.15.3:0/3869764552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc106f10 0x7f26fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 --2- 172.21.15.3:0/3869764552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc102790 0x7f26fc1067e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 --2- 172.21.15.3:0/3869764552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc101de0 0x7f26fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 -- 172.21.15.3:0/3869764552 >> 172.21.15.3:0/3869764552 conn(0x7f26fc0fd8c0 msgr2=0x7f26fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 -- 172.21.15.3:0/3869764552 shutdown_connections 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.410+0000 7f27043ee640 1 -- 172.21.15.3:0/3869764552 wait complete. 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 Processor -- start 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 -- start start 2024-09-12T02:19:47.456 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc102790 0x7f26fc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc106f10 0x7f26fc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc111f30 con 0x7f26fc101de0 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc111db0 con 0x7f26fc106f10 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f27043ee640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1120b0 con 0x7f26fc102790 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33460/0 (socket says 172.21.15.3:33460) 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f270218a640 1 -- 172.21.15.3:0/2288389815 learned_addr learned my addr 172.21.15.3:0/2288389815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f270298b640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc106f10 0x7f26fc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.411+0000 7f2701989640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc102790 0x7f26fc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 -- 172.21.15.3:0/2288389815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc102790 msgr2=0x7f26fc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc102790 0x7f26fc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 -- 172.21.15.3:0/2288389815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc106f10 msgr2=0x7f26fc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc106f10 0x7f26fc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 -- 172.21.15.3:0/2288389815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc19d3e0 con 0x7f26fc101de0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f270218a640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26f800eab0 tx=0x7f26f800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f26f37fe640 1 -- 172.21.15.3:0/2288389815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f800ce40 con 0x7f26fc101de0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f26f37fe640 1 -- 172.21.15.3:0/2288389815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f8011950 con 0x7f26fc101de0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f26f37fe640 1 -- 172.21.15.3:0/2288389815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8010ae0 con 0x7f26fc101de0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc19d6d0 con 0x7f26fc101de0 2024-09-12T02:19:47.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.412+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc1a4fb0 con 0x7f26fc101de0 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.415+0000 7f26f37fe640 1 -- 172.21.15.3:0/2288389815 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f26f8011b00 con 0x7f26fc101de0 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.415+0000 7f27067c7740 1 Processor -- start 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.415+0000 7f27067c7740 1 -- start start 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d5b40 0x560c4c4d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d45d0 0x560c4c4d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560c4c43a2f0 con 0x560c4c4e3780 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560c4c3e1cf0 con 0x560c4c4d45d0 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f27067c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560c4c4362b0 con 0x560c4c4d5b40 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33476/0 (socket says 172.21.15.3:33476) 2024-09-12T02:19:47.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 -- 172.21.15.3:0/2991956749 learned_addr learned my addr 172.21.15.3:0/2991956749 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f270298b640 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d5b40 0x560c4c4d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f270218a640 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d45d0 0x560c4c4d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f26f37fe640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 0x7f26e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 -- 172.21.15.3:0/2991956749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d5b40 msgr2=0x560c4c4d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d5b40 0x560c4c4d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 -- 172.21.15.3:0/2991956749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d45d0 msgr2=0x560c4c4d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d45d0 0x560c4c4d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f2701989640 1 -- 172.21.15.3:0/2991956749 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4c3c63a0 con 0x560c4c4e3780 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f26f37fe640 1 -- 172.21.15.3:0/2288389815 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f26f80cf750 con 0x7f26fc101de0 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.416+0000 7f270298b640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 0x7f26e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.417+0000 7f2701989640 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26ec02a930 tx=0x7f26ec005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d45ed2efeab245 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.417+0000 7f26f0dea640 1 -- 172.21.15.3:0/2991956749 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec03f070 con 0x560c4c4e3780 2024-09-12T02:19:47.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.417+0000 7f26f0dea640 1 -- 172.21.15.3:0/2991956749 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec004030 con 0x560c4c4e3780 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.417+0000 7f26f0dea640 1 -- 172.21.15.3:0/2991956749 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec035070 con 0x560c4c4e3780 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- 172.21.15.3:0/2991956749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 msgr2=0x560c4c4e5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26ec02a930 tx=0x7f26ec005dd0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f270298b640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 0x7f26e80a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26f40058c0 tx=0x7f26f40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- 172.21.15.3:0/2991956749 shutdown_connections 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4e5b70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d45d0 0x560c4c4d4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 --2- 172.21.15.3:0/2991956749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d5b40 0x560c4c4d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- 172.21.15.3:0/2991956749 >> 172.21.15.3:0/2991956749 conn(0x560c4c49d310 msgr2=0x560c4c4d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- 172.21.15.3:0/2991956749 shutdown_connections 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- 172.21.15.3:0/2991956749 wait complete. 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 -- start start 2024-09-12T02:19:47.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.418+0000 7f27067c7740 1 Processor -- start 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560c4c4e6420 con 0x560c4c4d45d0 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33488/0 (socket says 172.21.15.3:33488) 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560c4c3f5060 con 0x560c4c4d5b40 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 -- 172.21.15.3:0/4268487375 learned_addr learned my addr 172.21.15.3:0/4268487375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f2701989640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560c4c4e65a0 con 0x560c4c4e3780 2024-09-12T02:19:47.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f27067c7740 10 client.0 _ll_register_callbacks cb 0x560c4c1488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270298b640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 -- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 msgr2=0x560c4c4e09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4e09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 -- 172.21.15.3:0/4268487375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 msgr2=0x560c4c4e0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4e0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 -- 172.21.15.3:0/4268487375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4c1ab3c0 con 0x560c4c4d45d0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f270218a640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f26f8098910 tx=0x7f26f80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285bbb229b49bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f26e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f809c640 con 0x560c4c4d45d0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.419+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f26f809c1a0 con 0x560c4c4d45d0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ec004030 con 0x560c4c4d45d0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26f809bf40 con 0x560c4c4d45d0 2024-09-12T02:19:47.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c4c4f1ed0 con 0x560c4c4d45d0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ec004030 con 0x560c4c4d45d0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f26f800f040 con 0x560c4c4d45d0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 10 client.15918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 20 client.15918 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c4f33e0 0x560c4c4f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 4 client.15918 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c4c447860 con 0x560c4c4f33e0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.420+0000 7f2701989640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c4f33e0 0x560c4c4f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f2701989640 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c4f33e0 0x560c4c4f57d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb63c5da591961c7 server_cookie=7b5790f0767cbdc8 in_seq=0 out_seq=0 2024-09-12T02:19:47.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f26e5ffb640 10 client.15918 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f26e5ffb640 1 -- 172.21.15.3:0/4268487375 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c4c447860 con 0x560c4c4f33e0 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f26e5ffb640 10 client.15918 handle_command_reply: tid=0 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f27067c7740 1 client.15918 shutdown 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 msgr2=0x560c4c4e3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.421+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f26f8098910 tx=0x7f26f80103d0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 shutdown_connections 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c4f33e0 0x560c4c4f57d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f26e5ffb640 0 client.15918 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 --2- 172.21.15.3:0/4268487375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4e3150 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 >> 172.21.15.3:0/4268487375 conn(0x560c4c49d310 msgr2=0x560c4c4d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 shutdown_connections 2024-09-12T02:19:47.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 1 -- 172.21.15.3:0/4268487375 wait complete. 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.422+0000 7f27067c7740 20 client.15918 trim_cache size 0 max 16384 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 Processor -- start 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 -- start start 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d45d0 0x560c4c4ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4d5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ab2e0 con 0x560c4c4e3780 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ace20 con 0x560c4c4d5b40 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f27067c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ab9b0 con 0x560c4c4d45d0 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f270298b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f270298b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33498/0 (socket says 172.21.15.3:33498) 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f270298b640 1 -- 172.21.15.3:0/2159274734 learned_addr learned my addr 172.21.15.3:0/2159274734 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:47.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.435+0000 7f270218a640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d45d0 0x560c4c4ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f2701989640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4d5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 -- 172.21.15.3:0/2159274734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d45d0 msgr2=0x560c4c4ef6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d45d0 0x560c4c4ef6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 -- 172.21.15.3:0/2159274734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 msgr2=0x560c4c4d5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4d5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 -- 172.21.15.3:0/2159274734 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4c4e0d60 con 0x560c4c4e3780 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f2701989640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4d5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f270298b640 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26f401d610 tx=0x7f26f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1832eed68ef127fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f26e7fff640 1 -- 172.21.15.3:0/2159274734 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f402e070 con 0x560c4c4e3780 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f26e7fff640 1 -- 172.21.15.3:0/2159274734 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f401cb80 con 0x560c4c4e3780 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.436+0000 7f26e7fff640 1 -- 172.21.15.3:0/2159274734 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4029cd0 con 0x560c4c4e3780 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- 172.21.15.3:0/2159274734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 msgr2=0x560c4c4dfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26f401d610 tx=0x7f26f4019000 comp rx=0 tx=0).stop 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- 172.21.15.3:0/2159274734 shutdown_connections 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4e3780 0x560c4c4dfcc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4d5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- 172.21.15.3:0/2159274734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4d45d0 0x560c4c4ef6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- 172.21.15.3:0/2159274734 >> 172.21.15.3:0/2159274734 conn(0x560c4c49d310 msgr2=0x560c4c4decd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- 172.21.15.3:0/2159274734 shutdown_connections 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- 172.21.15.3:0/2159274734 wait complete. 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- start start 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 Processor -- start 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ac750 con 0x560c4c4d45d0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1ac080 con 0x560c4c4d5b40 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1a92c0 con 0x560c4c4e3780 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.437+0000 7f27067c7740 10 client.0 _ll_register_callbacks cb 0x7f26f809be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33514/0 (socket says 172.21.15.3:33514) 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 -- 172.21.15.3:0/534239885 learned_addr learned my addr 172.21.15.3:0/534239885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f2701989640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270298b640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 -- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 msgr2=0x560c4c4dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4dcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 -- 172.21.15.3:0/534239885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 msgr2=0x560c4c4ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 -- 172.21.15.3:0/534239885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4c4dd6a0 con 0x560c4c4d45d0 2024-09-12T02:19:47.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f270218a640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26f80baba0 tx=0x7f26f80cd260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a811d0efb4d84b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f26e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f809c640 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f26f809c1a0 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f401cb80 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26f80a02f0 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f27067c7740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.438+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c4c55dbc0 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f401cb80 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f26f809b500 con 0x560c4c4d45d0 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 10 client.15924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 20 client.15924 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:47.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c55e2e0 0x560c4c5606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 4 client.15924 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c4c441d70 con 0x560c4c55e2e0 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f2701989640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c55e2e0 0x560c4c5606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f2701989640 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c55e2e0 0x560c4c5606d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e38a13536dd5a9e server_cookie=bd84cc6c3d016c74 in_seq=0 out_seq=0 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.439+0000 7f26e5ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f26e5ffb640 1 -- 172.21.15.3:0/534239885 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c4c441d70 con 0x560c4c55e2e0 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f26e5ffb640 10 client.15924 handle_command_reply: tid=0 2024-09-12T02:19:47.472 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 client.15924 shutdown 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 msgr2=0x560c4c4ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26f80baba0 tx=0x7f26f80cd260 comp rx=0 tx=0).stop 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 shutdown_connections 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560c4c55e2e0 0x560c4c5606d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560c4c4e3780 0x560c4c4dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f26e5ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c4c4d5b40 0x560c4c4ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 --2- 172.21.15.3:0/534239885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560c4c4d45d0 0x560c4c4ebae0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 >> 172.21.15.3:0/534239885 conn(0x560c4c49d310 msgr2=0x560c4c4cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 shutdown_connections 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 1 -- 172.21.15.3:0/534239885 wait complete. 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.440+0000 7f27067c7740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 msgr2=0x7f26e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 0x7f26e80a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f26f40058c0 tx=0x7f26f40046c0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 msgr2=0x7f26fc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26f800eab0 tx=0x7f26f800ef70 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 shutdown_connections 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f26e80a2c00 0x7f26e80a50c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26fc106f10 0x7f26fc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26fc102790 0x7f26fc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 --2- 172.21.15.3:0/2288389815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f26fc101de0 0x7f26fc198430 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:47.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 >> 172.21.15.3:0/2288389815 conn(0x7f26fc0fd8c0 msgr2=0x7f26fc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:47.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 shutdown_connections 2024-09-12T02:19:47.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:47.441+0000 7f27043ee640 1 -- 172.21.15.3:0/2288389815 wait complete. 2024-09-12T02:19:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1645261552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0100a40 msgr2=0x7fa1e0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1645261552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0100a40 0x7fa1e0100e40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009900 tx=0x7fa1dc02a410 comp rx=0 tx=0).stop 2024-09-12T02:19:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1645261552 shutdown_connections 2024-09-12T02:19:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1645261552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1645261552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0101c30 0x7fa1e01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1645261552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0100a40 0x7fa1e0100e40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1645261552 >> 172.21.15.3:0/1645261552 conn(0x7fa1e00fc1c0 msgr2=0x7fa1e00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1645261552 shutdown_connections 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.070+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1645261552 wait complete. 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 Processor -- start 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 -- start start 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 0x7fa1e0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e010d870 con 0x7fa1e0101c30 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e010d6f0 con 0x7fa1e0100a40 2024-09-12T02:19:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e82b7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e010d9f0 con 0x7fa1e01025f0 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e6053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 0x7fa1e0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33540/0 (socket says 172.21.15.3:33540) 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e5852640 1 -- 172.21.15.3:0/1251660363 learned_addr learned my addr 172.21.15.3:0/1251660363 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.071+0000 7fa1e6854640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 -- 172.21.15.3:0/1251660363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 msgr2=0x7fa1e019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 -- 172.21.15.3:0/1251660363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 msgr2=0x7fa1e0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 0x7fa1e0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 -- 172.21.15.3:0/1251660363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e019fab0 con 0x7fa1e0101c30 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e6053640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 0x7fa1e0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e6854640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e5852640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000d950 tx=0x7fa1d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1d77fe640 1 -- 172.21.15.3:0/1251660363 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0017030 con 0x7fa1e0101c30 2024-09-12T02:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e019fda0 con 0x7fa1e0101c30 2024-09-12T02:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1d77fe640 1 -- 172.21.15.3:0/1251660363 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d0004020 con 0x7fa1e0101c30 2024-09-12T02:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1d77fe640 1 -- 172.21.15.3:0/1251660363 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d00047a0 con 0x7fa1e0101c30 2024-09-12T02:19:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.072+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e01a0260 con 0x7fa1e0101c30 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1d77fe640 1 -- 172.21.15.3:0/1251660363 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa1d00041d0 con 0x7fa1e0101c30 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 Processor -- start 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 -- start start 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab12e0 0x55ec5bab5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec5ba142f0 con 0x55ec5bab6370 2024-09-12T02:19:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec5b9bc270 con 0x55ec5bab12e0 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec5ba102b0 con 0x55ec5bab09a0 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e6854640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e6854640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:60248/0 (socket says 172.21.15.3:60248) 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33542/0 (socket says 172.21.15.3:33542) 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e6854640 1 -- 172.21.15.3:0/711467825 learned_addr learned my addr 172.21.15.3:0/711467825 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.075+0000 7fa1e6053640 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab12e0 0x55ec5bab5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1d77fe640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 -- 172.21.15.3:0/711467825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab12e0 msgr2=0x55ec5bab5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab12e0 0x55ec5bab5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 -- 172.21.15.3:0/711467825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 msgr2=0x55ec5bab8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 -- 172.21.15.3:0/711467825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5b9bb340 con 0x55ec5bab09a0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e5852640 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1d77fe640 1 -- 172.21.15.3:0/1251660363 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa1d00cea00 con 0x7fa1e0101c30 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80118d0 tx=0x7fa1d8011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7c0063240c1cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1d4bfe640 1 -- 172.21.15.3:0/711467825 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8017050 con 0x55ec5bab09a0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1d4bfe640 1 -- 172.21.15.3:0/711467825 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8026440 con 0x55ec5bab09a0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1e6854640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8023000 tx=0x7fa1d8024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.076+0000 7fa1d4bfe640 1 -- 172.21.15.3:0/711467825 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800e5b0 con 0x55ec5bab09a0 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- 172.21.15.3:0/711467825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 msgr2=0x55ec5bab0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80118d0 tx=0x7fa1d8011d90 comp rx=0 tx=0).stop 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- 172.21.15.3:0/711467825 shutdown_connections 2024-09-12T02:19:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bab8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab12e0 0x55ec5bab5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/711467825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab09a0 0x55ec5bab0da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- 172.21.15.3:0/711467825 >> 172.21.15.3:0/711467825 conn(0x55ec5baac010 msgr2=0x55ec5baac410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- 172.21.15.3:0/711467825 shutdown_connections 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- 172.21.15.3:0/711467825 wait complete. 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.077+0000 7fa1eb04e740 1 -- start start 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 Processor -- start 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ec5bab9340 con 0x55ec5bab6370 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ec5b99d290 con 0x55ec5bab09a0 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ec5bab94c0 con 0x55ec5bab12e0 2024-09-12T02:19:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1eb04e740 10 client.0 _ll_register_callbacks cb 0x55ec5b9ce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e6854640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e6053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:60254/0 (socket says 172.21.15.3:60254) 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e6053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38238/0 (socket says 172.21.15.3:38238) 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e6854640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33546/0 (socket says 172.21.15.3:33546) 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 -- 172.21.15.3:0/2803827709 learned_addr learned my addr 172.21.15.3:0/2803827709 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 -- 172.21.15.3:0/2803827709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 msgr2=0x55ec5babab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 -- 172.21.15.3:0/2803827709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 msgr2=0x55ec5bac2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 -- 172.21.15.3:0/2803827709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5b727210 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e6854640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1e5852640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00a3990 tx=0x7fa1d0005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a4a37e75411651a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.078+0000 7fa1c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d009ad90 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa1d00bb0c0 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8026440 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1d009abb0 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 10 client.25561 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5b7f3620 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8026440 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1d009f170 con 0x55ec5bab12e0 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 10 client.25561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 20 client.25561 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bac6850 0x55ec5bac8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 4 client.25561 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec5ba21a20 con 0x55ec5bac6850 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.079+0000 7fa1e6053640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bac6850 0x55ec5bac8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.080+0000 7fa1e6053640 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bac6850 0x55ec5bac8c40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68755f9a41dcef8 server_cookie=5b19c2a8040fa09b in_seq=0 out_seq=0 2024-09-12T02:19:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.080+0000 7fa1c9ffb640 10 client.25561 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/2803827709 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec5ba21a20 con 0x55ec5bac6850 2024-09-12T02:19:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1c9ffb640 10 client.25561 handle_command_reply: tid=0 2024-09-12T02:19:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 client.25561 shutdown 2024-09-12T02:19:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 msgr2=0x55ec5babb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00a3990 tx=0x7fa1d0005cd0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 shutdown_connections 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bac6850 0x55ec5bac8c40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab6370 0x55ec5bac2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1c9ffb640 0 client.25561 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5babb0d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2803827709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 >> 172.21.15.3:0/2803827709 conn(0x55ec5baac010 msgr2=0x55ec5baad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 shutdown_connections 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2803827709 wait complete. 2024-09-12T02:19:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.081+0000 7fa1eb04e740 20 client.25561 trim_cache size 0 max 16384 2024-09-12T02:19:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.096+0000 7fa1eb04e740 1 Processor -- start 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 -- start start 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5bac20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab6370 0x55ec5baac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01a9110 con 0x55ec5bab09a0 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01aac50 con 0x55ec5bab6370 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01a97e0 con 0x55ec5bab12e0 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33550/0 (socket says 172.21.15.3:33550) 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 -- 172.21.15.3:0/2814094072 learned_addr learned my addr 172.21.15.3:0/2814094072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e5852640 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5bac20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6854640 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab6370 0x55ec5baac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 -- 172.21.15.3:0/2814094072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 msgr2=0x55ec5bac20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5bac20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 -- 172.21.15.3:0/2814094072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab6370 msgr2=0x55ec5baac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab6370 0x55ec5baac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 -- 172.21.15.3:0/2814094072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5bab17c0 con 0x55ec5bab09a0 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1e6053640 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009900 tx=0x7fa1dc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c0397e333a0a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.097+0000 7fa1cbfff640 1 -- 172.21.15.3:0/2814094072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc04c070 con 0x55ec5bab09a0 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1cbfff640 1 -- 172.21.15.3:0/2814094072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc004c30 con 0x55ec5bab09a0 2024-09-12T02:19:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1cbfff640 1 -- 172.21.15.3:0/2814094072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc035040 con 0x55ec5bab09a0 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2814094072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 msgr2=0x55ec5baac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009900 tx=0x7fa1dc009260 comp rx=0 tx=0).stop 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2814094072 shutdown_connections 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab6370 0x55ec5baac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab12e0 0x55ec5bac20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/2814094072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab09a0 0x55ec5baac3f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.098+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2814094072 >> 172.21.15.3:0/2814094072 conn(0x55ec5baac010 msgr2=0x55ec5babab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2814094072 shutdown_connections 2024-09-12T02:19:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- 172.21.15.3:0/2814094072 wait complete. 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- start start 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 Processor -- start 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 0x55ec5bac06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01aa580 con 0x55ec5bab12e0 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01a9eb0 con 0x55ec5bab09a0 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1eb04e740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01a70f0 con 0x55ec5bab6370 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1e5852640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33564/0 (socket says 172.21.15.3:33564) 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1e5852640 1 -- 172.21.15.3:0/3589423963 learned_addr learned my addr 172.21.15.3:0/3589423963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1e6854640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 0x55ec5bac06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.099+0000 7fa1e6053640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1eb04e740 10 client.0 _ll_register_callbacks cb 0x7fa1d00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 -- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 msgr2=0x55ec5bac06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 0x55ec5bac06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 -- 172.21.15.3:0/3589423963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 msgr2=0x55ec5babf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 -- 172.21.15.3:0/3589423963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec5bac0cc0 con 0x55ec5bab12e0 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e6854640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 0x55ec5bac06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1e5852640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1d009f570 tx=0x7fa1d0004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e0cbd8674d29b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d009ad90 con 0x55ec5bab12e0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa1d009a8a0 con 0x55ec5bab12e0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1dc004c30 con 0x55ec5bab12e0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1d009b950 con 0x55ec5bab12e0 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1eb04e740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.100+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec5bac2040 con 0x55ec5bab12e0 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc004c30 con 0x55ec5bab12e0 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa1d009a110 con 0x55ec5bab12e0 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 20 client.15942 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bb31890 0x55ec5bb33cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 4 client.15942 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec5ba1bd70 con 0x55ec5bb31890 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1e6053640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bb31890 0x55ec5bb33cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1c9ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.101+0000 7fa1e6053640 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bb31890 0x55ec5bb33cb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b88cc60b64cce0 server_cookie=7551b538395c0c6b in_seq=0 out_seq=0 2024-09-12T02:19:48.154 INFO:tasks.workunit.client.0.smithi003.stderr:+ trap - EXIT 2024-09-12T02:19:48.154 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd ffsb 2024-09-12T02:19:48.155 INFO:tasks.workunit.client.0.smithi003.stderr:+ ./configure 2024-09-12T02:19:48.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:19:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1c9ffb640 1 -- 172.21.15.3:0/3589423963 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec5ba1bd70 con 0x55ec5bb31890 2024-09-12T02:19:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1c9ffb640 10 client.15942 handle_command_reply: tid=0 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 client.15942 shutdown 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 msgr2=0x55ec5babf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1d009f570 tx=0x7fa1d0004540 comp rx=0 tx=0).stop 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 shutdown_connections 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ec5bb31890 0x55ec5bb33cb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ec5bab6370 0x55ec5bac06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ec5bab12e0 0x55ec5babf6f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 --2- 172.21.15.3:0/3589423963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec5bab09a0 0x55ec5babf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1c9ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 >> 172.21.15.3:0/3589423963 conn(0x55ec5baac010 msgr2=0x55ec5baaa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 shutdown_connections 2024-09-12T02:19:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 1 -- 172.21.15.3:0/3589423963 wait complete. 2024-09-12T02:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.336+0000 7fa1eb04e740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T02:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:19:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 msgr2=0x7fa1cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8023000 tx=0x7fa1d8024000 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 msgr2=0x7fa1e01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000d950 tx=0x7fa1d000de10 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 shutdown_connections 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa1cc0a2b80 0x7fa1cc0a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa1e01025f0 0x7fa1e019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1e0101c30 0x7fa1e01983d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 --2- 172.21.15.3:0/1251660363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1e0100a40 0x7fa1e0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 >> 172.21.15.3:0/1251660363 conn(0x7fa1e00fc1c0 msgr2=0x7fa1e0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 shutdown_connections 2024-09-12T02:19:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:19:48.337+0000 7fa1e82b7640 1 -- 172.21.15.3:0/1251660363 wait complete. 2024-09-12T02:19:48.340 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T02:19:48.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:48 smithi003 ceph-mon[31476]: pgmap v378: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:48.566 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:48 smithi026 ceph-mon[35496]: pgmap v378: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:48.578 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:48.590 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:48 smithi092 ceph-mon[36089]: pgmap v378: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-12T02:19:49.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 -- 172.21.15.3:0/3748697835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0100a80 msgr2=0x7f8ad0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/3748697835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0100a80 0x7f8ad0100ee0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009930 tx=0x7f8ac402a420 comp rx=0 tx=0).stop 2024-09-12T02:19:49.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 -- 172.21.15.3:0/3748697835 shutdown_connections 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/3748697835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0101420 0x7f8ad010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/3748697835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0100a80 0x7f8ad0100ee0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/3748697835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0106a80 0x7f8ad0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 -- 172.21.15.3:0/3748697835 >> 172.21.15.3:0/3748697835 conn(0x7f8ad00fc7a0 msgr2=0x7f8ad00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 -- 172.21.15.3:0/3748697835 shutdown_connections 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.078+0000 7f8ad68a4640 1 -- 172.21.15.3:0/3748697835 wait complete. 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 Processor -- start 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 -- start start 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0100a80 0x7f8ad0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0106a80 0x7f8ad019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad0110e80 con 0x7f8ad0101420 2024-09-12T02:19:49.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad0110d00 con 0x7f8ad0100a80 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.079+0000 7f8ad68a4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad0111000 con 0x7f8ad0106a80 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47934/0 (socket says 172.21.15.3:47934) 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 -- 172.21.15.3:0/724374921 learned_addr learned my addr 172.21.15.3:0/724374921 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acffff640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0100a80 0x7f8ad0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0106a80 0x7f8ad019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 -- 172.21.15.3:0/724374921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0106a80 msgr2=0x7f8ad019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0106a80 0x7f8ad019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 -- 172.21.15.3:0/724374921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0100a80 msgr2=0x7f8ad0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0100a80 0x7f8ad0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 -- 172.21.15.3:0/724374921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad019d3e0 con 0x7f8ad0101420 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009230 tx=0x7f8ac4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acd7fa640 1 -- 172.21.15.3:0/724374921 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac403f030 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad019d670 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acd7fa640 1 -- 172.21.15.3:0/724374921 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4003ad0 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.080+0000 7f8acd7fa640 1 -- 172.21.15.3:0/724374921 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4035070 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.081+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad01a4fb0 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8acd7fa640 1 -- 172.21.15.3:0/724374921 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8ac4042030 con 0x7f8ad0101420 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 Processor -- start 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 -- start start 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f66b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d339ebb2f0 con 0x55d339f56b40 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d339e62cf0 con 0x55d339f64780 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d339eb72b0 con 0x55d339f555d0 2024-09-12T02:19:49.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.083+0000 7f8ad4e41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47948/0 (socket says 172.21.15.3:47948) 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 -- 172.21.15.3:0/3177313534 learned_addr learned my addr 172.21.15.3:0/3177313534 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8acffff640 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 -- 172.21.15.3:0/3177313534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 msgr2=0x55d339f55a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f55a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 -- 172.21.15.3:0/3177313534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 msgr2=0x55d339f66b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f66b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 -- 172.21.15.3:0/3177313534 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d339e473a0 con 0x55d339f56b40 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8acd7fa640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 0x7f8ab80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009ee0 tx=0x7f8ac8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c5074885aef303 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8acffff640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 0x7f8ab80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8acd7fa640 1 -- 172.21.15.3:0/724374921 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8ac40eee90 con 0x7f8ad0101420 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ab67fc640 1 -- 172.21.15.3:0/3177313534 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8016070 con 0x55d339f56b40 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ab67fc640 1 -- 172.21.15.3:0/3177313534 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8004d20 con 0x55d339f56b40 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.084+0000 7f8ab67fc640 1 -- 172.21.15.3:0/3177313534 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80127b0 con 0x55d339f56b40 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/3177313534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 msgr2=0x55d339f55090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8009ee0 tx=0x7f8ac8010740 comp rx=0 tx=0).stop 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8acffff640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 0x7f8ab80a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d339f52ac0 tx=0x7f8ac000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/3177313534 shutdown_connections 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f66b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f55a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/3177313534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f55090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/3177313534 >> 172.21.15.3:0/3177313534 conn(0x55d339f1e310 msgr2=0x55d339f51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/3177313534 shutdown_connections 2024-09-12T02:19:49.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/3177313534 wait complete. 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.085+0000 7f8ad8c7d740 1 -- start start 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 Processor -- start 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f64160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d339f67480 con 0x55d339f56b40 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55d339e76060 con 0x55d339f64780 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d339f67600 con 0x55d339f555d0 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad8c7d740 10 client.0 _ll_register_callbacks cb 0x55d339bc98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47950/0 (socket says 172.21.15.3:47950) 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 -- 172.21.15.3:0/2708086652 learned_addr learned my addr 172.21.15.3:0/2708086652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:49.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acffff640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f64160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 -- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 msgr2=0x55d339f64160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f64160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 -- 172.21.15.3:0/2708086652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 msgr2=0x55d339f619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8acf7fe640 1 -- 172.21.15.3:0/2708086652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d339c2c3c0 con 0x55d339f56b40 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.086+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40bb3e0 tx=0x7f8ac402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87f4f297a52ef87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4049050 con 0x55d339f56b40 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8ac4003c80 con 0x55d339f56b40 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac8004d20 con 0x55d339f56b40 2024-09-12T02:19:49.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ac40bb040 con 0x55d339f56b40 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d339f72f00 con 0x55d339f56b40 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac8004d20 con 0x55d339f56b40 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ac40b7d50 con 0x55d339f56b40 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 20 client.15954 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339f742a0 0x55d339f76690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 4 client.15954 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.087+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d339ec8860 con 0x55d339f742a0 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.088+0000 7f8acffff640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339f742a0 0x55d339f76690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.088+0000 7f8acffff640 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339f742a0 0x55d339f76690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eabfed7312003f2c server_cookie=e3dd0b3c7aa8f0db in_seq=0 out_seq=0 2024-09-12T02:19:49.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.088+0000 7f8aaf7fe640 10 client.15954 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/2708086652 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d339ec8860 con 0x55d339f742a0 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8aaf7fe640 10 client.15954 handle_command_reply: tid=0 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 client.15954 shutdown 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 msgr2=0x55d339f61410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40bb3e0 tx=0x7f8ac402a420 comp rx=0 tx=0).stop 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 shutdown_connections 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339f742a0 0x55d339f76690 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f56b40 0x55d339f61410 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8aaf7fe640 0 client.15954 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/2708086652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f555d0 0x55d339f64160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 >> 172.21.15.3:0/2708086652 conn(0x55d339f1e310 msgr2=0x55d339f57960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 shutdown_connections 2024-09-12T02:19:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/2708086652 wait complete. 2024-09-12T02:19:49.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.089+0000 7f8ad8c7d740 20 client.15954 trim_cache size 0 max 16384 2024-09-12T02:19:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 Processor -- start 2024-09-12T02:19:49.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 -- start start 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f70720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ab2f0 con 0x55d339f555d0 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ace30 con 0x55d339f64780 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ab9c0 con 0x55d339f56b40 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8acffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8acffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47962/0 (socket says 172.21.15.3:47962) 2024-09-12T02:19:49.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.103+0000 7f8acffff640 1 -- 172.21.15.3:0/1514659261 learned_addr learned my addr 172.21.15.3:0/1514659261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f70720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 -- 172.21.15.3:0/1514659261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 msgr2=0x55d339f70720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f70720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 -- 172.21.15.3:0/1514659261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 msgr2=0x55d339f7bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f7bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 -- 172.21.15.3:0/1514659261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d339f58120 con 0x55d339f555d0 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f70720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f7bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:49.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8acffff640 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d339f76ed0 tx=0x7f8ac0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8863b14ef08f0548 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8ab5ffb640 1 -- 172.21.15.3:0/1514659261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac002c070 con 0x55d339f555d0 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8ab5ffb640 1 -- 172.21.15.3:0/1514659261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac001bdf0 con 0x55d339f555d0 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.104+0000 7f8ab5ffb640 1 -- 172.21.15.3:0/1514659261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0028410 con 0x55d339f555d0 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/1514659261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 msgr2=0x55d339f51e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x55d339f76ed0 tx=0x7f8ac0019000 comp rx=0 tx=0).stop 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/1514659261 shutdown_connections 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f64780 0x55d339f7bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f70720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/1514659261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f555d0 0x55d339f51e10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/1514659261 >> 172.21.15.3:0/1514659261 conn(0x55d339f1e310 msgr2=0x55d339f51700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/1514659261 shutdown_connections 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/1514659261 wait complete. 2024-09-12T02:19:49.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 -- start start 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.105+0000 7f8ad8c7d740 1 Processor -- start 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 0x55d339f68620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f68b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ac760 con 0x55d339f64780 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01ac090 con 0x55d339f555d0 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad01a92f0 con 0x55d339f56b40 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad8c7d740 10 client.0 _ll_register_callbacks cb 0x7f8ac4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad4e41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f68b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 0x55d339f68620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acf7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f68b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37188/0 (socket says 172.21.15.3:37188) 2024-09-12T02:19:49.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acf7fe640 1 -- 172.21.15.3:0/837441311 learned_addr learned my addr 172.21.15.3:0/837441311 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad4e41640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47966/0 (socket says 172.21.15.3:47966) 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 -- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 msgr2=0x55d339f68b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f68b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 -- 172.21.15.3:0/837441311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 msgr2=0x55d339f690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 -- 172.21.15.3:0/837441311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d339fddf30 con 0x55d339f555d0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8ad4e41640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8acffff640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 0x55d339f68620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0011010 tx=0x7f8ac00035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feaee6019e4559f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.106+0000 7f8aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac002c070 con 0x55d339f555d0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac001bdf0 con 0x55d339f555d0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8ac0005240 con 0x55d339f555d0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ac0005440 con 0x55d339f555d0 2024-09-12T02:19:49.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 10 client.25484 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d339fdf160 con 0x55d339f555d0 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac001bdf0 con 0x55d339f555d0 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ac0003f60 con 0x55d339f555d0 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 10 client.25484 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 10 client.25484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 20 client.25484 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339fdf7c0 0x55d339fe1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 4 client.25484 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.107+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d339ec2d70 con 0x55d339fdf7c0 2024-09-12T02:19:49.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339fdf7c0 0x55d339fe1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:49.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8aaf7fe640 10 client.25484 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:49.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8acf7fe640 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339fdf7c0 0x55d339fe1bb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82670abb6c491c5b server_cookie=6253d4c8d21fc88f in_seq=0 out_seq=0 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8aaf7fe640 1 -- 172.21.15.3:0/837441311 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d339ec2d70 con 0x55d339fdf7c0 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8aaf7fe640 10 client.25484 handle_command_reply: tid=0 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 client.25484 shutdown 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 msgr2=0x55d339f68620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 0x55d339f68620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0011010 tx=0x7f8ac00035f0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 shutdown_connections 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x55d339fdf7c0 0x55d339fe1bb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d339f64780 0x55d339f690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8aaf7fe640 0 client.25484 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d339f56b40 0x55d339f68b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 --2- 172.21.15.3:0/837441311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d339f555d0 0x55d339f68620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 >> 172.21.15.3:0/837441311 conn(0x55d339f1e310 msgr2=0x55d339f62c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 shutdown_connections 2024-09-12T02:19:49.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.108+0000 7f8ad8c7d740 1 -- 172.21.15.3:0/837441311 wait complete. 2024-09-12T02:19:49.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad8c7d740 20 client.25484 trim_cache size 0 max 16384 2024-09-12T02:19:49.116 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:49.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 msgr2=0x7f8ab80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.116 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 0x7f8ab80a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d339f52ac0 tx=0x7f8ac000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 msgr2=0x7f8ad0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009230 tx=0x7f8ac4004270 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 shutdown_connections 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8ab80a2b80 0x7f8ab80a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad0106a80 0x7f8ad019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ad0101420 0x7f8ad0198970 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 --2- 172.21.15.3:0/724374921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ad0100a80 0x7f8ad0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 >> 172.21.15.3:0/724374921 conn(0x7f8ad00fc7a0 msgr2=0x7f8ad0104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 shutdown_connections 2024-09-12T02:19:49.117 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:49.109+0000 7f8ad68a4640 1 -- 172.21.15.3:0/724374921 wait complete. 2024-09-12T02:19:49.345 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether build environment is sane... yes 2024-09-12T02:19:49.353 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T02:19:49.353 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gawk... gawk 2024-09-12T02:19:49.368 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T02:19:49.397 INFO:tasks.workunit.client.0.smithi003.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T02:19:49.398 INFO:tasks.workunit.client.0.smithi003.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T02:19:49.408 INFO:tasks.workunit.client.0.smithi003.stdout:checking for style of include used by make... GNU 2024-09-12T02:19:49.412 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... gcc 2024-09-12T02:19:49.463 INFO:tasks.workunit.client.0.smithi003.stdout:checking for C compiler default output file name... a.out 2024-09-12T02:19:49.465 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the C compiler works... yes 2024-09-12T02:19:49.467 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are cross compiling... no 2024-09-12T02:19:49.504 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of executables... 2024-09-12T02:19:49.536 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of object files... o 2024-09-12T02:19:49.567 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T02:19:49.597 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... yes 2024-09-12T02:19:49.753 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T02:19:49.851 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... gcc3 2024-09-12T02:19:49.863 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:19:49.933 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T02:19:49.975 INFO:tasks.workunit.client.0.smithi003.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T02:19:49.976 INFO:tasks.workunit.client.0.smithi003.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T02:19:49.985 INFO:tasks.workunit.client.0.smithi003.stdout:checking for AIX... no 2024-09-12T02:19:50.004 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... (cached) gcc 2024-09-12T02:19:50.013 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T02:19:50.014 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T02:19:50.014 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T02:19:50.014 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T02:19:50.069 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lm... yes 2024-09-12T02:19:50.119 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lpthread... yes 2024-09-12T02:19:50.176 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:50.300 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... yes 2024-09-12T02:19:50.459 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T02:19:50.531 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... yes 2024-09-12T02:19:50.592 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... yes 2024-09-12T02:19:50.594 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:50 smithi003 ceph-mon[31476]: pgmap v379: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-09-12T02:19:50.638 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdlib.h... yes 2024-09-12T02:19:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 -- 172.21.15.3:0/4126913000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102ef0 msgr2=0x7fd73c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 --2- 172.21.15.3:0/4126913000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102ef0 0x7fd73c10da40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd734009ca0 tx=0x7fd73402a950 comp rx=0 tx=0).stop 2024-09-12T02:19:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 -- 172.21.15.3:0/4126913000 shutdown_connections 2024-09-12T02:19:50.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 --2- 172.21.15.3:0/4126913000 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102ef0 0x7fd73c10da40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 --2- 172.21.15.3:0/4126913000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102570 0x7fd73c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 --2- 172.21.15.3:0/4126913000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 -- 172.21.15.3:0/4126913000 >> 172.21.15.3:0/4126913000 conn(0x7fd73c0fc7d0 msgr2=0x7fd73c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 -- 172.21.15.3:0/4126913000 shutdown_connections 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.682+0000 7fd741a92640 1 -- 172.21.15.3:0/4126913000 wait complete. 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 Processor -- start 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 -- start start 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 0x7fd73c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.683+0000 7fd741a92640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c110f80 con 0x7fd73c102570 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd741a92640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c110e00 con 0x7fd73c102ef0 2024-09-12T02:19:50.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd741a92640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c111100 con 0x7fd73c100e20 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47994/0 (socket says 172.21.15.3:47994) 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 -- 172.21.15.3:0/1555234764 learned_addr learned my addr 172.21.15.3:0/1555234764 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 0x7fd73c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 -- 172.21.15.3:0/1555234764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 msgr2=0x7fd73c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 -- 172.21.15.3:0/1555234764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 msgr2=0x7fd73c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 0x7fd73c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 -- 172.21.15.3:0/1555234764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c19d370 con 0x7fd73c102570 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 0x7fd73c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:50.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd73affd640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c920 tx=0x7fd73000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd738ff9640 1 -- 172.21.15.3:0/1555234764 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730007c70 con 0x7fd73c102570 2024-09-12T02:19:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd738ff9640 1 -- 172.21.15.3:0/1555234764 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd730007e10 con 0x7fd73c102570 2024-09-12T02:19:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd738ff9640 1 -- 172.21.15.3:0/1555234764 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730018890 con 0x7fd73c102570 2024-09-12T02:19:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c19d660 con 0x7fd73c102570 2024-09-12T02:19:50.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.684+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1a4f40 con 0x7fd73c102570 2024-09-12T02:19:50.688 INFO:tasks.workunit.client.0.smithi003.stdout:checking for string.h... yes 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.687+0000 7fd738ff9640 1 -- 172.21.15.3:0/1555234764 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd730018a70 con 0x7fd73c102570 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.687+0000 7fd743e6b740 1 Processor -- start 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.687+0000 7fd743e6b740 1 -- start start 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ec3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ed0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2e292f0 con 0x55a1c2ece370 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2dd0cf0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.694 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd743e6b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2e252b0 con 0x55a1c2ec4b40 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ed0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37224/0 (socket says 172.21.15.3:37224) 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ec3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73bfff640 1 -- 172.21.15.3:0/1478970957 learned_addr learned my addr 172.21.15.3:0/1478970957 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd738ff9640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 0x7fd7240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1478970957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 msgr2=0x55a1c2ec3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1478970957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 msgr2=0x55a1c2ed0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ed0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1478970957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c2db53a0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:50.695 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd738ff9640 1 -- 172.21.15.3:0/1555234764 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd7300cf1a0 con 0x7fd73c102570 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.688+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 0x7fd7240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.689+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ec3a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009fd0 tx=0x7fd72c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca891a7b3431f8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.689+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 0x7fd7240a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd734009430 tx=0x7fd7340095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.689+0000 7fd721ffb640 1 -- 172.21.15.3:0/1478970957 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004030 con 0x55a1c2ec35d0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.689+0000 7fd721ffb640 1 -- 172.21.15.3:0/1478970957 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c005500 con 0x55a1c2ec35d0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.689+0000 7fd721ffb640 1 -- 172.21.15.3:0/1478970957 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c004d50 con 0x55a1c2ec35d0 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- 172.21.15.3:0/1478970957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 msgr2=0x55a1c2ec3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ec3a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009fd0 tx=0x7fd72c0118d0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- 172.21.15.3:0/1478970957 shutdown_connections 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ed0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ec3a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1478970957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- 172.21.15.3:0/1478970957 >> 172.21.15.3:0/1478970957 conn(0x55a1c2e8c310 msgr2=0x55a1c2ebfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- 172.21.15.3:0/1478970957 shutdown_connections 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- 172.21.15.3:0/1478970957 wait complete. 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 -- start start 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.690+0000 7fd743e6b740 1 Processor -- start 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ecafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ecb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2ed1070 con 0x55a1c2ec35d0 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2de4060 con 0x55a1c2ece370 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a1c2ed11f0 con 0x55a1c2ec4b40 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd743e6b740 10 client.0 _ll_register_callbacks cb 0x55a1c2b378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:50.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48018/0 (socket says 172.21.15.3:48018) 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 -- 172.21.15.3:0/4133967043 learned_addr learned my addr 172.21.15.3:0/4133967043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73affd640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ecafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73bfff640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ecb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 -- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 msgr2=0x55a1c2ecafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ecafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 -- 172.21.15.3:0/4133967043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 msgr2=0x55a1c2ecb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ecb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 -- 172.21.15.3:0/4133967043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c2b9a3c0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009350 tx=0x7fd72c00fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f5f5985fc81bea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.691+0000 7fd71affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c028070 con 0x55a1c2ec35d0 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd72c004c60 con 0x55a1c2ec35d0 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c005500 con 0x55a1c2ec35d0 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd72c034050 con 0x55a1c2ec35d0 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:50.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1c2edcaa0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72c005500 con 0x55a1c2ec35d0 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd72c019570 con 0x55a1c2ec35d0 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 20 client.15972 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:50.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2eddeb0 0x55a1c2ee02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 4 client.15972 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.692+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1c2e36860 con 0x55a1c2eddeb0 2024-09-12T02:19:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.693+0000 7fd73affd640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2eddeb0 0x55a1c2ee02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.693+0000 7fd71affd640 10 client.15972 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:50.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.693+0000 7fd73affd640 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2eddeb0 0x55a1c2ee02a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1ae7ca18661e2b server_cookie=16d72eb6371b3d56 in_seq=0 out_seq=0 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.694+0000 7fd71affd640 1 -- 172.21.15.3:0/4133967043 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1c2e36860 con 0x55a1c2eddeb0 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.694+0000 7fd71affd640 10 client.15972 handle_command_reply: tid=0 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.694+0000 7fd743e6b740 1 client.15972 shutdown 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 msgr2=0x55a1c2ecdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009350 tx=0x7fd72c00fca0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 shutdown_connections 2024-09-12T02:19:50.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2eddeb0 0x55a1c2ee02a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ecb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ecafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 --2- 172.21.15.3:0/4133967043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ecdd00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd71affd640 0 client.15972 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 >> 172.21.15.3:0/4133967043 conn(0x55a1c2e8c310 msgr2=0x55a1c2ec5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 shutdown_connections 2024-09-12T02:19:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 1 -- 172.21.15.3:0/4133967043 wait complete. 2024-09-12T02:19:50.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.695+0000 7fd743e6b740 20 client.15972 trim_cache size 0 max 16384 2024-09-12T02:19:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 Processor -- start 2024-09-12T02:19:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 -- start start 2024-09-12T02:19:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2eda310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.709 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ec6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ad5d0 con 0x55a1c2ec4b40 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1af110 con 0x55a1c2ece370 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd743e6b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1adca0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd73affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd73affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.709+0000 7fd73affd640 1 -- 172.21.15.3:0/339313672 learned_addr learned my addr 172.21.15.3:0/339313672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:50.710 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2eda310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73bfff640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ec6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 -- 172.21.15.3:0/339313672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 msgr2=0x55a1c2eda310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2eda310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 -- 172.21.15.3:0/339313672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 msgr2=0x55a1c2ec6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ec6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 -- 172.21.15.3:0/339313672 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1c2ec0730 con 0x55a1c2ec4b40 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73bfff640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ec6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd73affd640 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd730098360 tx=0x7fd730086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fac6dd1c43e952 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd7217fa640 1 -- 172.21.15.3:0/339313672 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73009b990 con 0x55a1c2ec4b40 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd7217fa640 1 -- 172.21.15.3:0/339313672 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73009fd40 con 0x55a1c2ec4b40 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.710+0000 7fd7217fa640 1 -- 172.21.15.3:0/339313672 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730098000 con 0x55a1c2ec4b40 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 -- 172.21.15.3:0/339313672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 msgr2=0x55a1c2ec6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd730098360 tx=0x7fd730086040 comp rx=0 tx=0).stop 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 -- 172.21.15.3:0/339313672 shutdown_connections 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ece370 0x55a1c2ec6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec6120 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 --2- 172.21.15.3:0/339313672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2eda310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 -- 172.21.15.3:0/339313672 >> 172.21.15.3:0/339313672 conn(0x55a1c2e8c310 msgr2=0x55a1c2ebed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 -- 172.21.15.3:0/339313672 shutdown_connections 2024-09-12T02:19:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.711+0000 7fd743e6b740 1 -- 172.21.15.3:0/339313672 wait complete. 2024-09-12T02:19:50.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 -- start start 2024-09-12T02:19:50.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 Processor -- start 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ec8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1aea40 con 0x55a1c2ece370 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ae370 con 0x55a1c2ec4b40 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.712+0000 7fd743e6b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c1ab5b0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd743e6b740 10 client.0 _ll_register_callbacks cb 0x7fd730097c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37252/0 (socket says 172.21.15.3:37252) 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1549720485 learned_addr learned my addr 172.21.15.3:0/1549720485 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73bfff640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ec8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73affd640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1549720485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec4b40 msgr2=0x55a1c2ec7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1549720485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 msgr2=0x55a1c2ec8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ec8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 -- 172.21.15.3:0/1549720485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1c2ec8a00 con 0x55a1c2ec35d0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd73b7fe640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009380 tx=0x7fd72c004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7d238099591048 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd71affd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c028040 con 0x55a1c2ec35d0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73009fd40 con 0x55a1c2ec35d0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.713+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fd72c019450 con 0x55a1c2ec35d0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd72c005d30 con 0x55a1c2ec35d0 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 10 client.25603 fetch_fsmap learned FSMap version 29 2024-09-12T02:19:50.718 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1c2f488f0 con 0x55a1c2ec35d0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd73009fd40 con 0x55a1c2ec35d0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd72c041050 con 0x55a1c2ec35d0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 10 client.25603 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 20 client.25603 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2f48f80 0x55a1c2f4b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 4 client.25603 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1c2e30d70 con 0x55a1c2f48f80 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.714+0000 7fd73affd640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2f48f80 0x55a1c2f4b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd71affd640 10 client.25603 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd73affd640 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2f48f80 0x55a1c2f4b3a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0fb1390039909f1 server_cookie=5dcb6a4a34340b7 in_seq=0 out_seq=0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd71affd640 1 -- 172.21.15.3:0/1549720485 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1c2e30d70 con 0x55a1c2f48f80 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd71affd640 10 client.25603 handle_command_reply: tid=0 2024-09-12T02:19:50.719 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 client.25603 shutdown 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 msgr2=0x55a1c2ed6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009380 tx=0x7fd72c004990 comp rx=0 tx=0).stop 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 shutdown_connections 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55a1c2f48f80 0x55a1c2f4b3a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a1c2ece370 0x55a1c2ec8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a1c2ec4b40 0x55a1c2ec7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 --2- 172.21.15.3:0/1549720485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a1c2ec35d0 0x55a1c2ed6b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd71affd640 0 client.25603 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 >> 172.21.15.3:0/1549720485 conn(0x55a1c2e8c310 msgr2=0x55a1c2ec9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 shutdown_connections 2024-09-12T02:19:50.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 1 -- 172.21.15.3:0/1549720485 wait complete. 2024-09-12T02:19:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.715+0000 7fd743e6b740 20 client.25603 trim_cache size 0 max 16384 2024-09-12T02:19:50.721 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:19:50.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 msgr2=0x7fd7240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 0x7fd7240a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd734009430 tx=0x7fd7340095c0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 msgr2=0x7fd73c198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c920 tx=0x7fd73000cde0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 shutdown_connections 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fd7240a2c00 0x7fd7240a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd73c102ef0 0x7fd73c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd73c102570 0x7fd73c198900 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 --2- 172.21.15.3:0/1555234764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd73c100e20 0x7fd73c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 >> 172.21.15.3:0/1555234764 conn(0x7fd73c0fc7d0 msgr2=0x7fd73c06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 shutdown_connections 2024-09-12T02:19:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:50.716+0000 7fd741a92640 1 -- 172.21.15.3:0/1555234764 wait complete. 2024-09-12T02:19:50.735 INFO:tasks.workunit.client.0.smithi003.stdout:checking for memory.h... yes 2024-09-12T02:19:50.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:50 smithi026 ceph-mon[35496]: pgmap v379: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-09-12T02:19:50.784 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... yes 2024-09-12T02:19:50.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:50 smithi092 ceph-mon[36089]: pgmap v379: 97 pgs: 97 active+clean; 672 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 17 op/s 2024-09-12T02:19:50.851 INFO:tasks.workunit.client.0.smithi003.stdout:checking for inttypes.h... yes 2024-09-12T02:19:50.898 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... yes 2024-09-12T02:19:50.949 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... yes 2024-09-12T02:19:51.044 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h usability... yes 2024-09-12T02:19:51.068 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h presence... yes 2024-09-12T02:19:51.068 INFO:tasks.workunit.client.0.smithi003.stdout:checking for pthread.h... yes 2024-09-12T02:19:51.128 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h usability... yes 2024-09-12T02:19:51.150 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h presence... yes 2024-09-12T02:19:51.151 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fcntl.h... yes 2024-09-12T02:19:51.213 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h usability... yes 2024-09-12T02:19:51.233 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h presence... yes 2024-09-12T02:19:51.233 INFO:tasks.workunit.client.0.smithi003.stdout:checking for limits.h... yes 2024-09-12T02:19:51.248 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... (cached) yes 2024-09-12T02:19:51.304 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h usability... yes 2024-09-12T02:19:51.338 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h presence... yes 2024-09-12T02:19:51.338 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/time.h... yes 2024-09-12T02:19:51.349 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... (cached) yes 2024-09-12T02:19:51.412 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h usability... yes 2024-09-12T02:19:51.432 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h presence... yes 2024-09-12T02:19:51.432 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/vfs.h... yes 2024-09-12T02:19:51.518 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h usability... no 2024-09-12T02:19:51.539 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h presence... no 2024-09-12T02:19:51.539 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/limits.h... no 2024-09-12T02:19:51.571 INFO:tasks.workunit.client.0.smithi003.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T02:19:51.605 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T02:19:51.606 INFO:tasks.workunit.client.0.smithi003.stdout:checking for function prototypes... yes 2024-09-12T02:19:51.633 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether setvbuf arguments are reversed... no 2024-09-12T02:19:51.709 INFO:tasks.workunit.client.0.smithi003.stdout:checking for system... yes 2024-09-12T02:19:51.772 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gettimeofday... yes 2024-09-12T02:19:51.833 INFO:tasks.workunit.client.0.smithi003.stdout:checking for mkdir... yes 2024-09-12T02:19:51.897 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strerror... yes 2024-09-12T02:19:51.959 INFO:tasks.workunit.client.0.smithi003.stdout:checking for open64... yes 2024-09-12T02:19:52.021 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stat64... yes 2024-09-12T02:19:52.083 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fseeko64... yes 2024-09-12T02:19:52.144 INFO:tasks.workunit.client.0.smithi003.stdout:checking for lrand48_r... yes 2024-09-12T02:19:52.207 INFO:tasks.workunit.client.0.smithi003.stdout:checking for srand48_r... yes 2024-09-12T02:19:52.235 INFO:tasks.workunit.client.0.smithi003.stdout:configure: creating ./config.status 2024-09-12T02:19:52.341 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating Makefile 2024-09-12T02:19:52.355 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-09-12T02:19:52.372 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing depfiles commands 2024-09-12T02:19:52.420 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing default commands 2024-09-12T02:19:52.466 INFO:tasks.workunit.client.0.smithi003.stderr:+ make 2024-09-12T02:19:52.473 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-12T02:19:52.475 INFO:tasks.workunit.client.0.smithi003.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-12T02:19:52.476 INFO:tasks.workunit.client.0.smithi003.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-12T02:19:52.476 INFO:tasks.workunit.client.0.smithi003.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-12T02:19:52.476 INFO:tasks.workunit.client.0.smithi003.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-12T02:19:52.476 INFO:tasks.workunit.client.0.smithi003.stderr: any GNU archive site. 2024-09-12T02:19:52.478 INFO:tasks.workunit.client.0.smithi003.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-12T02:19:52.479 INFO:tasks.workunit.client.0.smithi003.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-12T02:19:52.480 INFO:tasks.workunit.client.0.smithi003.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-12T02:19:52.480 INFO:tasks.workunit.client.0.smithi003.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-12T02:19:52.480 INFO:tasks.workunit.client.0.smithi003.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-12T02:19:52.480 INFO:tasks.workunit.client.0.smithi003.stderr: Grab them from any GNU archive site. 2024-09-12T02:19:52.530 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-12T02:19:52.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:19:52.554 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:19:52.594 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:52 smithi003 ceph-mon[31476]: pgmap v380: 97 pgs: 97 active+clean; 2.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 141 KiB/s wr, 47 op/s 2024-09-12T02:19:52.595 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:52 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:52.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:52 smithi026 ceph-mon[35496]: pgmap v380: 97 pgs: 97 active+clean; 2.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 141 KiB/s wr, 47 op/s 2024-09-12T02:19:52.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:52 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:52.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:52 smithi092 ceph-mon[36089]: pgmap v380: 97 pgs: 97 active+clean; 2.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 141 KiB/s wr, 47 op/s 2024-09-12T02:19:52.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:52 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:52.834 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:52.870 INFO:tasks.workunit.client.0.smithi003.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-12T02:19:52.870 INFO:tasks.workunit.client.0.smithi003.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-12T02:19:52.870 INFO:tasks.workunit.client.0.smithi003.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-12T02:19:52.870 INFO:tasks.workunit.client.0.smithi003.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-12T02:19:52.989 INFO:tasks.workunit.client.0.smithi003.stdout:/bin/sh ./config.status --recheck 2024-09-12T02:19:53.043 INFO:tasks.workunit.client.0.smithi003.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-12T02:19:53.185 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 -- 172.21.15.3:0/2807863857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c102d90 msgr2=0x7f159c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2807863857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c102d90 0x7f159c103170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f158c009cd0 tx=0x7f158c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 -- 172.21.15.3:0/2807863857 shutdown_connections 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2807863857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c108290 0x7f159c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2807863857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2807863857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c102d90 0x7f159c103170 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 -- 172.21.15.3:0/2807863857 >> 172.21.15.3:0/2807863857 conn(0x7f159c0fc7d0 msgr2=0x7f159c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 -- 172.21.15.3:0/2807863857 shutdown_connections 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 -- 172.21.15.3:0/2807863857 wait complete. 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.341+0000 7f15a1b66640 1 Processor -- start 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 -- start start 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c102d90 0x7f159c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f159c112090 con 0x7f159c108290 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f159c111f10 con 0x7f159c102d90 2024-09-12T02:19:53.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f15a1b66640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c112210 con 0x7f159c103740 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48052/0 (socket says 172.21.15.3:48052) 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 -- 172.21.15.3:0/2271680557 learned_addr learned my addr 172.21.15.3:0/2271680557 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159affd640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159b7fe640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c102d90 0x7f159c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 -- 172.21.15.3:0/2271680557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 msgr2=0x7f159c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 -- 172.21.15.3:0/2271680557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c102d90 msgr2=0x7f159c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c102d90 0x7f159c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159bfff640 1 -- 172.21.15.3:0/2271680557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c19d500 con 0x7f159c108290 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.342+0000 7f159affd640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f159bfff640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f159400c970 tx=0x7f159400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f1598ff9640 1 -- 172.21.15.3:0/2271680557 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594007bb0 con 0x7f159c108290 2024-09-12T02:19:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c19d7f0 con 0x7f159c108290 2024-09-12T02:19:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f1598ff9640 1 -- 172.21.15.3:0/2271680557 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1594007d50 con 0x7f159c108290 2024-09-12T02:19:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f1598ff9640 1 -- 172.21.15.3:0/2271680557 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594018800 con 0x7f159c108290 2024-09-12T02:19:53.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.343+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1a50d0 con 0x7f159c108290 2024-09-12T02:19:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 Processor -- start 2024-09-12T02:19:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f1598ff9640 1 -- 172.21.15.3:0/2271680557 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f159400fa50 con 0x7f159c108290 2024-09-12T02:19:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 -- start start 2024-09-12T02:19:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5d1d40 0x562aad5d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562aad52c280 con 0x562aad5c6b60 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aad4d3b10 con 0x562aad5d1d40 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aad528240 con 0x562aad5c7480 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48062/0 (socket says 172.21.15.3:48062) 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.346+0000 7f159affd640 1 -- 172.21.15.3:0/3349543667 learned_addr learned my addr 172.21.15.3:0/3349543667 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159b7fe640 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5d1d40 0x562aad5d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 -- 172.21.15.3:0/3349543667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 msgr2=0x562aad5d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 -- 172.21.15.3:0/3349543667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5d1d40 msgr2=0x562aad5d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5d1d40 0x562aad5d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f1598ff9640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 0x7f15840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 -- 172.21.15.3:0/3349543667 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aad4e01f0 con 0x562aad5c6b60 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159b7fe640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 0x7f15840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f1598ff9640 1 -- 172.21.15.3:0/2271680557 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15940cefb0 con 0x7f159c108290 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f159affd640 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f159000bfd0 tx=0x7f159000bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797554f22665f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f1581ffb640 1 -- 172.21.15.3:0/3349543667 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590005020 con 0x562aad5c6b60 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f1581ffb640 1 -- 172.21.15.3:0/3349543667 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15900056a0 con 0x562aad5c6b60 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.347+0000 7f1581ffb640 1 -- 172.21.15.3:0/3349543667 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590004a00 con 0x562aad5c6b60 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3349543667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 msgr2=0x562aad5c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f159b7fe640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 0x7f15840a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x562aad5c3f50 tx=0x7f158c0314f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f159000bfd0 tx=0x7f159000bbb0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3349543667 shutdown_connections 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5d1d40 0x562aad5d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3349543667 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5c6f40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3349543667 >> 172.21.15.3:0/3349543667 conn(0x562aad5c30f0 msgr2=0x562aad5c34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3349543667 shutdown_connections 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3349543667 wait complete. 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.348+0000 7f15a3f3f740 1 -- start start 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 Processor -- start 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c7480 0x562aad5cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562aad5d4910 con 0x562aad5c6b60 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562aad4e6f10 con 0x562aad5c7480 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562aad5d4a90 con 0x562aad5d1d40 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f15a3f3f740 10 client.0 _ll_register_callbacks cb 0x562aad4afa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48070/0 (socket says 172.21.15.3:48070) 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 -- 172.21.15.3:0/3050781490 learned_addr learned my addr 172.21.15.3:0/3050781490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159bfff640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159affd640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c7480 0x562aad5cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 -- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 msgr2=0x562aad5cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 -- 172.21.15.3:0/3050781490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c7480 msgr2=0x562aad5cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c7480 0x562aad5cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.349+0000 7f159b7fe640 1 -- 172.21.15.3:0/3050781490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aad29d6a0 con 0x562aad5c6b60 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f159bfff640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f159b7fe640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f158c035000 tx=0x7f158c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115529dc970a42ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f157affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c054060 con 0x562aad5c6b60 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f158c004540 con 0x562aad5c6b60 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15900056a0 con 0x562aad5c6b60 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f158c004740 con 0x562aad5c6b60 2024-09-12T02:19:53.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f15a3f3f740 10 client.15990 fetch_fsmap learned FSMap version 31 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562aad5e04c0 con 0x562aad5c6b60 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.350+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15900056a0 con 0x562aad5c6b60 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f158c04e030 con 0x562aad5c6b60 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 10 client.15990 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 20 client.15990 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad5e19d0 0x562aad5e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 4 client.15990 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:53.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aad5397f0 con 0x562aad5e19d0 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f159affd640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad5e19d0 0x562aad5e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f159affd640 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad5e19d0 0x562aad5e3de0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff17fefa046dbbb6 server_cookie=372ea04ecf837fbd in_seq=0 out_seq=0 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.351+0000 7f157affd640 10 client.15990 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f157affd640 1 -- 172.21.15.3:0/3050781490 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562aad5397f0 con 0x562aad5e19d0 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f157affd640 10 client.15990 handle_command_reply: tid=0 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 client.15990 shutdown 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 msgr2=0x562aad5cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f158c035000 tx=0x7f158c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 shutdown_connections 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad5e19d0 0x562aad5e3de0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c7480 0x562aad5cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/3050781490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c6b60 0x562aad5cebd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 >> 172.21.15.3:0/3050781490 conn(0x562aad5c30f0 msgr2=0x562aad5c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.352+0000 7f157affd640 0 client.15990 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.353+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 shutdown_connections 2024-09-12T02:19:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.353+0000 7f15a3f3f740 1 -- 172.21.15.3:0/3050781490 wait complete. 2024-09-12T02:19:53.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.353+0000 7f15a3f3f740 20 client.15990 trim_cache size 0 max 16384 2024-09-12T02:19:53.371 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 Processor -- start 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 -- start start 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ad4a0 con 0x562aad5d1d40 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1af0a0 con 0x562aad5c6b60 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.370+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1adb70 con 0x562aad5c7480 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48084/0 (socket says 172.21.15.3:48084) 2024-09-12T02:19:53.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 -- 172.21.15.3:0/781713488 learned_addr learned my addr 172.21.15.3:0/781713488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159affd640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159b7fe640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 -- 172.21.15.3:0/781713488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 msgr2=0x562aad5d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 -- 172.21.15.3:0/781713488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 msgr2=0x562aad5c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 -- 172.21.15.3:0/781713488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aad5c72b0 con 0x562aad5d1d40 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159b7fe640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f159bfff640 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f159400afd0 tx=0x7f1594097420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafdb373e02d0887 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f15817fa640 1 -- 172.21.15.3:0/781713488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159409fc50 con 0x562aad5d1d40 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f15817fa640 1 -- 172.21.15.3:0/781713488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15940a4510 con 0x562aad5d1d40 2024-09-12T02:19:53.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.371+0000 7f15817fa640 1 -- 172.21.15.3:0/781713488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159409acf0 con 0x562aad5d1d40 2024-09-12T02:19:53.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- 172.21.15.3:0/781713488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 msgr2=0x562aad5da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f159400afd0 tx=0x7f1594097420 comp rx=0 tx=0).stop 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- 172.21.15.3:0/781713488 shutdown_connections 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5d1d40 0x562aad5da0c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5c7480 0x562aad5d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/781713488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- 172.21.15.3:0/781713488 >> 172.21.15.3:0/781713488 conn(0x562aad5c30f0 msgr2=0x562aad5d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- 172.21.15.3:0/781713488 shutdown_connections 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- 172.21.15.3:0/781713488 wait complete. 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.372+0000 7f15a3f3f740 1 -- start start 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 Processor -- start 2024-09-12T02:19:53.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5e9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ae940 con 0x562aad5c7480 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ae270 con 0x562aad5c6b60 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f159c1ab480 con 0x562aad5d1d40 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f15a3f3f740 10 client.0 _ll_register_callbacks cb 0x7f158c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48100/0 (socket says 172.21.15.3:48100) 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 -- 172.21.15.3:0/543286470 learned_addr learned my addr 172.21.15.3:0/543286470 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:53.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159bfff640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5e9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159b7fe640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 -- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 msgr2=0x562aad5e9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5e9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 -- 172.21.15.3:0/543286470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 msgr2=0x562aad5cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.373+0000 7f159affd640 1 -- 172.21.15.3:0/543286470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aad64f6f0 con 0x562aad5c7480 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f159affd640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f159000ed80 tx=0x7f159000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817b9fd5b088c493 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159000e950 con 0x562aad5c7480 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15900120d0 con 0x562aad5c7480 2024-09-12T02:19:53.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15940a4510 con 0x562aad5c7480 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1590004020 con 0x562aad5c7480 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 10 client.15996 fetch_fsmap learned FSMap version 31 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562aad6508f0 con 0x562aad5c7480 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15940a4510 con 0x562aad5c7480 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f1590027070 con 0x562aad5c7480 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 10 client.15996 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 20 client.15996 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad650f50 0x562aad653370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:53.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 4 client.15996 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.374+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562aad533d00 con 0x562aad650f50 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.375+0000 7f159b7fe640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad650f50 0x562aad653370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.375+0000 7f159b7fe640 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad650f50 0x562aad653370 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c364b6de04b3cf16 server_cookie=5d1dcb36a99b4903 in_seq=0 out_seq=0 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.375+0000 7f157affd640 10 client.15996 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f157affd640 1 -- 172.21.15.3:0/543286470 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562aad533d00 con 0x562aad650f50 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f157affd640 10 client.15996 handle_command_reply: tid=0 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 client.15996 shutdown 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 msgr2=0x562aad5e9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f159000ed80 tx=0x7f159000edb0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 shutdown_connections 2024-09-12T02:19:53.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562aad650f50 0x562aad653370 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562aad5d1d40 0x562aad5e9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f157affd640 0 client.15996 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562aad5c7480 0x562aad5e9320 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 --2- 172.21.15.3:0/543286470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562aad5c6b60 0x562aad5cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 >> 172.21.15.3:0/543286470 conn(0x562aad5c30f0 msgr2=0x562aad5c2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 shutdown_connections 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 1 -- 172.21.15.3:0/543286470 wait complete. 2024-09-12T02:19:53.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.376+0000 7f15a3f3f740 20 client.15996 trim_cache size 0 max 16384 2024-09-12T02:19:53.383 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "return_code": 0, 2024-09-12T02:19:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_tag": "6451d376-3acd-4879-8936-f382a9a376d6", 2024-09-12T02:19:53.383 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": "asynchronous" 2024-09-12T02:19:53.383 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 msgr2=0x7f15840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 0x7f15840a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x562aad5c3f50 tx=0x7f158c0314f0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 msgr2=0x7f159c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f159400c970 tx=0x7f159400ce30 comp rx=0 tx=0).stop 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 shutdown_connections 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15840a2b80 0x7f15840a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f159c108290 0x7f159c19ce20 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f159c103740 0x7f159c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 --2- 172.21.15.3:0/2271680557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f159c102d90 0x7f159c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 >> 172.21.15.3:0/2271680557 conn(0x7f159c0fc7d0 msgr2=0x7f159c105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 shutdown_connections 2024-09-12T02:19:53.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:53.377+0000 7f15a1b66640 1 -- 172.21.15.3:0/2271680557 wait complete. 2024-09-12T02:19:53.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:53 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:53.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:53 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:53.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:53 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:53.789 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:19:53 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: 2024-09-12T02:19:53.403+0000 7f9e6f8d3640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000279 [2,head] /volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b auth v408 ap=1 s=361 n(v0 rc2024-09-12T02:19:47.620495+0000 b361 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={24928=0-4194304@1} caps={24928=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=24928 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55d56541f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000279:[<0x10000000278/902847fb04ba0435f838cf7cdb9b56c42fb61b v408>,<0x1000000021f/9a v770>,<0x10000000008/objects v238>,<0x10000000007/.git v274>,<0x10000000006/ffsb v140>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v34>]//[]","memoryvalue":"(3)0x10000000279:[<0x10000000278/902847fb04ba0435f838cf7cdb9b56c42fb61b v408>,<0x1000000021f/9a v770>,<0x10000000008/objects v305>,<0x10000000007/.git v1512>,<0x10000000006/ffsb v1302>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v16>,<0x100000001fa/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 v18>,<0x10000000001/sv_1 v43>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-12T02:19:53.789 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:19:53 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: 2024-09-12T02:19:53.426+0000 7f9e6f8d3640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000002a5 [...2,head] /volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin/ auth{0=1,4=1} v7 ap=2 RANDEPHEMERALPIN f(v0 m2024-09-12T02:19:47.920491+0000 1=1+0) n(v1 rc2024-09-12T02:19:47.924491+0000 b188 2=1+1)/n(v0 rc2024-09-12T02:19:47.920491+0000 2=1+1) (inest sync->mix g=0,4 scatter_wanted) caps={24928=pAsLsXsFs/pAsLsXsFsx@7} | dirtyscattered=0 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 randepin 0x55d56541f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000002a5:[<0x100000002a4/origin v4>,<0x100000002a3/remotes v4>,<0x100000002a2/refs v8>,<0x10000000008/logs v271>,<0x10000000007/.git v274>,<0x10000000006/ffsb v140>,<0x10000000005/tmp v4>,<0x100000001fb/client.0 v6>,<0x100000001fa/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 v22>,<0x10000000001/sv_1 v45>,<0x10000000000/qa v35>,<0x1/volumes v34>]//[]","memoryvalue":"(2)0x100000002a5:[<0x100000002a4/origin v4>,<0x100000002a3/remotes v10>,<0x100000002a2/refs v14>,<0x10000000008/logs v339>,<0x10000000007/.git v1512>,<0x10000000006/ffsb v1302>,<0x10000000005/tmp v11>,<0x100000001fb/client.0 v16>,<0x100000001fa/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372 v18>,<0x10000000001/sv_1 v43>,<0x10000000000/qa v33>,<0x1/volumes v30>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T02:19:47.920491+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T02:19:47.920491+0000 2=1+1)","memory_value.dirstat":"f(v0 m2024-09-12T02:19:47.920491+0000 1=1+0)","memory_value.rstat":"n(v1 rc2024-09-12T02:19:47.924491+0000 b188 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-12T02:19:54.063 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:19:54.191 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether build environment is sane... yes 2024-09-12T02:19:54.196 INFO:tasks.workunit.client.0.smithi003.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-12T02:19:54.196 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gawk... gawk 2024-09-12T02:19:54.210 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T02:19:54.240 INFO:tasks.workunit.client.0.smithi003.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T02:19:54.241 INFO:tasks.workunit.client.0.smithi003.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T02:19:54.252 INFO:tasks.workunit.client.0.smithi003.stdout:checking for style of include used by make... GNU 2024-09-12T02:19:54.256 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... gcc 2024-09-12T02:19:54.338 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether the C compiler works... yes 2024-09-12T02:19:54.339 INFO:tasks.workunit.client.0.smithi003.stdout:checking for C compiler default output file name... a.out 2024-09-12T02:19:54.372 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:19:54.382 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of executables... 2024-09-12T02:19:54.421 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are cross compiling... no 2024-09-12T02:19:54.457 INFO:tasks.workunit.client.0.smithi003.stdout:checking for suffix of object files... o 2024-09-12T02:19:54.490 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T02:19:54.524 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... yes 2024-09-12T02:19:54.580 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T02:19:54.687 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... gcc3 2024-09-12T02:19:54.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: pgmap v381: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 144 KiB/s wr, 64 op/s 2024-09-12T02:19:54.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub queued for path: / 2024-09-12T02:19:54.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.702 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: Scrub error on inode 0x100000002a5 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.703 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:54 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.739 INFO:tasks.workunit.client.0.smithi003.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: pgmap v381: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 144 KiB/s wr, 64 op/s 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub queued for path: / 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: Scrub error on inode 0x100000002a5 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:54 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.785 INFO:tasks.workunit.client.0.smithi003.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T02:19:54.786 INFO:tasks.workunit.client.0.smithi003.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T02:19:54.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: pgmap v381: 97 pgs: 97 active+clean; 2.1 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 144 KiB/s wr, 64 op/s 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub queued for path: / 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: Scrub error on inode 0x100000002a5 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin) see mds.f log and `damage ls` output for details 2024-09-12T02:19:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:54 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:54.931 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... yes 2024-09-12T02:19:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 -- 172.21.15.3:0/186877654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8102570 msgr2=0x7f9ed81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 --2- 172.21.15.3:0/186877654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8102570 0x7f9ed81029b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009900 tx=0x7f9ec802a480 comp rx=0 tx=0).stop 2024-09-12T02:19:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 -- 172.21.15.3:0/186877654 shutdown_connections 2024-09-12T02:19:54.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 --2- 172.21.15.3:0/186877654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 --2- 172.21.15.3:0/186877654 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8102570 0x7f9ed81029b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 --2- 172.21.15.3:0/186877654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8100e20 0x7f9ed8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 -- 172.21.15.3:0/186877654 >> 172.21.15.3:0/186877654 conn(0x7f9ed80fc7d0 msgr2=0x7f9ed80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 -- 172.21.15.3:0/186877654 shutdown_connections 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.925+0000 7f9edff02640 1 -- 172.21.15.3:0/186877654 wait complete. 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 Processor -- start 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 -- start start 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8102570 0x7f9ed8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8110f80 con 0x7f9ed8100e20 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8110e00 con 0x7f9ed8102570 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edff02640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed8111100 con 0x7f9ed8102ef0 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9ede49f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48120/0 (socket says 172.21.15.3:48120) 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 -- 172.21.15.3:0/3114949297 learned_addr learned my addr 172.21.15.3:0/3114949297 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9edd49d640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8102570 0x7f9ed8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 -- 172.21.15.3:0/3114949297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 msgr2=0x7f9ed819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 -- 172.21.15.3:0/3114949297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8102570 msgr2=0x7f9ed8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8102570 0x7f9ed8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.926+0000 7f9eddc9e640 1 -- 172.21.15.3:0/3114949297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed819d490 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9ede49f640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9eceffd640 1 -- 172.21.15.3:0/3114949297 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4017070 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed819d780 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9eceffd640 1 -- 172.21.15.3:0/3114949297 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4009c30 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9eceffd640 1 -- 172.21.15.3:0/3114949297 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400f6b0 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.927+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed81a5060 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.929+0000 7f9eceffd640 1 -- 172.21.15.3:0/3114949297 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9ed40109d0 con 0x7f9ed8100e20 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 Processor -- start 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 -- start start 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d162f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.935 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 0x559d16304a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559d1625d2e0 con 0x559d162f7dc0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d16204ce0 con 0x559d162f74a0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9ee22db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559d162592a0 con 0x559d16302680 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9eceffd640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 0x7f9ec40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 0x559d16304a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48128/0 (socket says 172.21.15.3:48128) 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 0x559d16304a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37332/0 (socket says 172.21.15.3:37332) 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.930+0000 7f9eddc9e640 1 -- 172.21.15.3:0/137154204 learned_addr learned my addr 172.21.15.3:0/137154204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9ede49f640 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d162f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9ede49f640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 0x7f9ec40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eceffd640 1 -- 172.21.15.3:0/3114949297 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9ed40cec70 con 0x7f9ed8100e20 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 -- 172.21.15.3:0/137154204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 msgr2=0x559d16304a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 0x559d16304a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 -- 172.21.15.3:0/137154204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 msgr2=0x559d162f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d162f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 -- 172.21.15.3:0/137154204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d161e9440 con 0x559d162f7dc0 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.931+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40cc640 tx=0x7f9ed4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63567fce7a694b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.932+0000 7f9ec3fff640 1 -- 172.21.15.3:0/137154204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4020030 con 0x559d162f7dc0 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.932+0000 7f9ede49f640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 0x7f9ec40a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005fd0 tx=0x7f9ed0011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.932+0000 7f9ec3fff640 1 -- 172.21.15.3:0/137154204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4009de0 con 0x559d162f7dc0 2024-09-12T02:19:54.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.932+0000 7f9ec3fff640 1 -- 172.21.15.3:0/137154204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4097b80 con 0x559d162f7dc0 2024-09-12T02:19:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- 172.21.15.3:0/137154204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 msgr2=0x559d16302140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40cc640 tx=0x7f9ed4005880 comp rx=0 tx=0).stop 2024-09-12T02:19:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- 172.21.15.3:0/137154204 shutdown_connections 2024-09-12T02:19:54.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d16302680 0x559d16304a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d162f7dc0 0x559d16302140 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 --2- 172.21.15.3:0/137154204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d162f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- 172.21.15.3:0/137154204 >> 172.21.15.3:0/137154204 conn(0x559d162f3a30 msgr2=0x559d162f3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- 172.21.15.3:0/137154204 shutdown_connections 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- 172.21.15.3:0/137154204 wait complete. 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 -- start start 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.933+0000 7f9ee22db740 1 Processor -- start 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d162ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559d16305240 con 0x559d16302680 2024-09-12T02:19:54.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559d16218050 con 0x559d162f7dc0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559d163053c0 con 0x559d162f74a0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ee22db740 10 client.0 _ll_register_callbacks cb 0x559d16258160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ede49f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37334/0 (socket says 172.21.15.3:37334) 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ede49f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162ff780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48140/0 (socket says 172.21.15.3:48140) 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 -- 172.21.15.3:0/1462337857 learned_addr learned my addr 172.21.15.3:0/1462337857 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9edd49d640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d162ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 -- 172.21.15.3:0/1462337857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 msgr2=0x559d162ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d162ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 -- 172.21.15.3:0/1462337857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 msgr2=0x559d162ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 -- 172.21.15.3:0/1462337857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d15fce3c0 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4010db0 tx=0x7f9ed40977b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c16ad57197993f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.934+0000 7f9ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4020040 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9ed409bc10 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed4009de0 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9ed409f810 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 10 client.25633 fetch_fsmap learned FSMap version 31 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d16305240 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4009de0 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9ed409b770 con 0x559d162f74a0 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 10 client.25633 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:54.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 20 client.25633 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16316150 0x559d16318540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 4 client.25633 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d1626a850 con 0x559d16316150 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.935+0000 7f9edd49d640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16316150 0x559d16318540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.936+0000 7f9ec17fa640 10 client.25633 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.936+0000 7f9edd49d640 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16316150 0x559d16318540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7d4aab996f61a9 server_cookie=dee6860be53e5899 in_seq=0 out_seq=0 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ec17fa640 1 -- 172.21.15.3:0/1462337857 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d1626a850 con 0x559d16316150 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ec17fa640 10 client.25633 handle_command_reply: tid=0 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 client.25633 shutdown 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 msgr2=0x559d16301f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4010db0 tx=0x7f9ed40977b0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 shutdown_connections 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16316150 0x559d16318540 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d162ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ec17fa640 0 client.25633 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 --2- 172.21.15.3:0/1462337857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d16301f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 >> 172.21.15.3:0/1462337857 conn(0x559d162f3a30 msgr2=0x559d162f9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 shutdown_connections 2024-09-12T02:19:54.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.938+0000 7f9ee22db740 1 -- 172.21.15.3:0/1462337857 wait complete. 2024-09-12T02:19:54.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.939+0000 7f9ee22db740 20 client.25633 trim_cache size 0 max 16384 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.956+0000 7f9ee22db740 1 Processor -- start 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.956+0000 7f9ee22db740 1 -- start start 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.956+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d1631dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.956+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d1631e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ee22db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81ab2f0 con 0x559d16302680 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ee22db740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81acef0 con 0x559d162f7dc0 2024-09-12T02:19:54.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ee22db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81ab9c0 con 0x559d162f74a0 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d1631e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ede49f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d1631dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d1631e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:35674/0 (socket says 172.21.15.3:35674) 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9edd49d640 1 -- 172.21.15.3:0/2282447612 learned_addr learned my addr 172.21.15.3:0/2282447612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 -- 172.21.15.3:0/2282447612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 msgr2=0x559d1631e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d1631e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 -- 172.21.15.3:0/2282447612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 msgr2=0x559d162fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 -- 172.21.15.3:0/2282447612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d162f3860 con 0x559d162f74a0 2024-09-12T02:19:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d1631dc30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40997c0 tx=0x7f9ed4099a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a46ee448c6e1cc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ec37fe640 1 -- 172.21.15.3:0/2282447612 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4020070 con 0x559d162f74a0 2024-09-12T02:19:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.957+0000 7f9ec37fe640 1 -- 172.21.15.3:0/2282447612 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed40a40d0 con 0x559d162f74a0 2024-09-12T02:19:54.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ec37fe640 1 -- 172.21.15.3:0/2282447612 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed409b770 con 0x559d162f74a0 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 -- 172.21.15.3:0/2282447612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 msgr2=0x559d1631dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d1631dc30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40997c0 tx=0x7f9ed4099a20 comp rx=0 tx=0).stop 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 -- 172.21.15.3:0/2282447612 shutdown_connections 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d162fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f7dc0 0x559d1631e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2282447612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f74a0 0x559d1631dc30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 -- 172.21.15.3:0/2282447612 >> 172.21.15.3:0/2282447612 conn(0x559d162f3a30 msgr2=0x559d162fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 -- 172.21.15.3:0/2282447612 shutdown_connections 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.958+0000 7f9ee22db740 1 -- 172.21.15.3:0/2282447612 wait complete. 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 -- start start 2024-09-12T02:19:54.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 Processor -- start 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d1630ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d1630a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81ac790 con 0x559d16302680 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81ac0c0 con 0x559d162f74a0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9ee22db740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed81a92d0 con 0x559d162f7dc0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.959+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ede49f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d1630a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ee22db740 10 client.0 _ll_register_callbacks cb 0x7f9ed4097070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37344/0 (socket says 172.21.15.3:37344) 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 -- 172.21.15.3:0/2129720151 learned_addr learned my addr 172.21.15.3:0/2129720151 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d1630ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 -- 172.21.15.3:0/2129720151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 msgr2=0x559d1630ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d1630ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 -- 172.21.15.3:0/2129720151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 msgr2=0x559d1630a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d1630a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 -- 172.21.15.3:0/2129720151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1630adc0 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9edd49d640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005940 tx=0x7f9ec80044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff8e421fab0b075 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ec17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8035040 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9ec8031950 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.960+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed40a40d0 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8031b50 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 10 client.25639 fetch_fsmap learned FSMap version 31 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d16380a30 con 0x559d162f7dc0 2024-09-12T02:19:54.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed40a40d0 con 0x559d162f7dc0 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f9ec8046310 con 0x559d162f7dc0 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 10 client.25639 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 10 client.25639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 20 client.25639 populate_metadata read hostname 'smithi003' 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16381270 0x559d16383690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 4 client.25639 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d16264d60 con 0x559d16381270 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.961+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16381270 0x559d16383690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ec17fa640 10 client.25639 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9eddc9e640 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16381270 0x559d16383690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82518aa4e5869244 server_cookie=720ecf870c7bcf43 in_seq=0 out_seq=0 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ec17fa640 1 -- 172.21.15.3:0/2129720151 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559d16264d60 con 0x559d16381270 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ec17fa640 10 client.25639 handle_command_reply: tid=0 2024-09-12T02:19:54.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 client.25639 shutdown 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 msgr2=0x559d1630a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005940 tx=0x7f9ec80044f0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 shutdown_connections 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559d16381270 0x559d16383690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559d16302680 0x559d1630a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559d162f7dc0 0x559d1630a2c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ec17fa640 0 client.25639 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 --2- 172.21.15.3:0/2129720151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559d162f74a0 0x559d1630ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 >> 172.21.15.3:0/2129720151 conn(0x559d162f3a30 msgr2=0x559d162f2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 shutdown_connections 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.962+0000 7f9ee22db740 1 -- 172.21.15.3:0/2129720151 wait complete. 2024-09-12T02:19:54.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9ee22db740 20 client.25639 trim_cache size 0 max 16384 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": { 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "6451d376-3acd-4879-8936-f382a9a376d6": { 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/", 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tag": "6451d376-3acd-4879-8936-f382a9a376d6", 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: "options": "recursive,force" 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:19:54.970 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:19:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 msgr2=0x7f9ec40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 0x7f9ec40a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005fd0 tx=0x7f9ed0011810 comp rx=0 tx=0).stop 2024-09-12T02:19:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 msgr2=0x7f9ed81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:19:54.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400d950 tx=0x7f9ed400de10 comp rx=0 tx=0).stop 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 shutdown_connections 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ec40a2b80 0x7f9ec40a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ed8102ef0 0x7f9ed819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed8102570 0x7f9ed8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 --2- 172.21.15.3:0/3114949297 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9ed8100e20 0x7f9ed81984e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 >> 172.21.15.3:0/3114949297 conn(0x7f9ed80fc7d0 msgr2=0x7f9ed806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 shutdown_connections 2024-09-12T02:19:54.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:19:54.963+0000 7f9edff02640 1 -- 172.21.15.3:0/3114949297 wait complete. 2024-09-12T02:19:54.972 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/types.h... yes 2024-09-12T02:19:55.014 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/stat.h... yes 2024-09-12T02:19:55.053 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdlib.h... yes 2024-09-12T02:19:55.095 INFO:tasks.workunit.client.0.smithi003.stdout:checking for string.h... yes 2024-09-12T02:19:55.134 INFO:tasks.workunit.client.0.smithi003.stdout:checking for memory.h... yes 2024-09-12T02:19:55.173 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strings.h... yes 2024-09-12T02:19:55.213 INFO:tasks.workunit.client.0.smithi003.stdout:checking for inttypes.h... yes 2024-09-12T02:19:55.256 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... yes 2024-09-12T02:19:55.297 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... yes 2024-09-12T02:19:55.348 INFO:tasks.workunit.client.0.smithi003.stdout:checking minix/config.h usability... no 2024-09-12T02:19:55.375 INFO:tasks.workunit.client.0.smithi003.stdout:checking minix/config.h presence... no 2024-09-12T02:19:55.375 INFO:tasks.workunit.client.0.smithi003.stdout:checking for minix/config.h... no 2024-09-12T02:19:55.416 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-12T02:19:55.416 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc... (cached) gcc 2024-09-12T02:19:55.449 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T02:19:55.450 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T02:19:55.450 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T02:19:55.450 INFO:tasks.workunit.client.0.smithi003.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-12T02:19:55.491 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lm... yes 2024-09-12T02:19:55.535 INFO:tasks.workunit.client.0.smithi003.stdout:checking for main in -lpthread... yes 2024-09-12T02:19:55.536 INFO:tasks.workunit.client.0.smithi003.stdout:checking for ANSI C header files... (cached) yes 2024-09-12T02:19:55.610 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-12T02:19:55.660 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h usability... yes 2024-09-12T02:19:55.679 INFO:tasks.workunit.client.0.smithi003.stdout:checking pthread.h presence... yes 2024-09-12T02:19:55.679 INFO:tasks.workunit.client.0.smithi003.stdout:checking for pthread.h... yes 2024-09-12T02:19:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:55 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:55 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:55 smithi003 ceph-mon[31476]: scrub summary: active paths [/] 2024-09-12T02:19:55.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:55 smithi003 ceph-mon[31476]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.719 INFO:tasks.cephfs.filesystem:scrub status for tag:6451d376-3acd-4879-8936-f382a9a376d6 - {'path': '/', 'tag': '6451d376-3acd-4879-8936-f382a9a376d6', 'options': 'recursive,force'} 2024-09-12T02:19:55.729 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h usability... yes 2024-09-12T02:19:55.745 INFO:tasks.workunit.client.0.smithi003.stdout:checking fcntl.h presence... yes 2024-09-12T02:19:55.745 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fcntl.h... yes 2024-09-12T02:19:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:55 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:55 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:55 smithi026 ceph-mon[35496]: scrub summary: active paths [/] 2024-09-12T02:19:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:55 smithi026 ceph-mon[35496]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:55 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:55 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:55 smithi092 ceph-mon[36089]: scrub summary: active paths [/] 2024-09-12T02:19:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:55 smithi092 ceph-mon[36089]: scrub summary: idle+waiting paths [/] 2024-09-12T02:19:55.810 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h usability... yes 2024-09-12T02:19:55.826 INFO:tasks.workunit.client.0.smithi003.stdout:checking limits.h presence... yes 2024-09-12T02:19:55.827 INFO:tasks.workunit.client.0.smithi003.stdout:checking for limits.h... yes 2024-09-12T02:19:55.840 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stdint.h... (cached) yes 2024-09-12T02:19:55.876 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h usability... yes 2024-09-12T02:19:55.893 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/time.h presence... yes 2024-09-12T02:19:55.893 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/time.h... yes 2024-09-12T02:19:55.912 INFO:tasks.workunit.client.0.smithi003.stdout:checking for unistd.h... (cached) yes 2024-09-12T02:19:55.953 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h usability... yes 2024-09-12T02:19:55.969 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/vfs.h presence... yes 2024-09-12T02:19:55.970 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/vfs.h... yes 2024-09-12T02:19:56.024 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h usability... no 2024-09-12T02:19:56.054 INFO:tasks.workunit.client.0.smithi003.stdout:checking sys/limits.h presence... no 2024-09-12T02:19:56.054 INFO:tasks.workunit.client.0.smithi003.stdout:checking for sys/limits.h... no 2024-09-12T02:19:56.080 INFO:tasks.workunit.client.0.smithi003.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T02:19:56.111 INFO:tasks.workunit.client.0.smithi003.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T02:19:56.175 INFO:tasks.workunit.client.0.smithi003.stdout:checking for system... yes 2024-09-12T02:19:56.230 INFO:tasks.workunit.client.0.smithi003.stdout:checking for gettimeofday... yes 2024-09-12T02:19:56.288 INFO:tasks.workunit.client.0.smithi003.stdout:checking for mkdir... yes 2024-09-12T02:19:56.340 INFO:tasks.workunit.client.0.smithi003.stdout:checking for strerror... yes 2024-09-12T02:19:56.396 INFO:tasks.workunit.client.0.smithi003.stdout:checking for open64... yes 2024-09-12T02:19:56.451 INFO:tasks.workunit.client.0.smithi003.stdout:checking for stat64... yes 2024-09-12T02:19:56.506 INFO:tasks.workunit.client.0.smithi003.stdout:checking for fseeko64... yes 2024-09-12T02:19:56.564 INFO:tasks.workunit.client.0.smithi003.stdout:checking for lrand48_r... yes 2024-09-12T02:19:56.622 INFO:tasks.workunit.client.0.smithi003.stdout:checking for srand48_r... yes 2024-09-12T02:19:56.649 INFO:tasks.workunit.client.0.smithi003.stdout:configure: creating ./config.status 2024-09-12T02:19:56.711 INFO:tasks.workunit.client.0.smithi003.stdout: /bin/sh ./config.status 2024-09-12T02:19:56.759 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating Makefile 2024-09-12T02:19:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:56 smithi026 ceph-mon[35496]: pgmap v382: 97 pgs: 97 active+clean; 2.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 189 KiB/s wr, 88 op/s 2024-09-12T02:19:56.771 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-09-12T02:19:56.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:56 smithi003 ceph-mon[31476]: pgmap v382: 97 pgs: 97 active+clean; 2.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 189 KiB/s wr, 88 op/s 2024-09-12T02:19:56.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:56 smithi092 ceph-mon[36089]: pgmap v382: 97 pgs: 97 active+clean; 2.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 189 KiB/s wr, 88 op/s 2024-09-12T02:19:56.794 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing depfiles commands 2024-09-12T02:19:56.806 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: executing default commands 2024-09-12T02:19:56.832 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-12T02:19:56.978 INFO:tasks.workunit.client.0.smithi003.stdout:rm -f stamp-h1 2024-09-12T02:19:56.981 INFO:tasks.workunit.client.0.smithi003.stdout:touch config.h.in 2024-09-12T02:19:56.982 INFO:tasks.workunit.client.0.smithi003.stdout:cd . && /bin/sh ./config.status config.h 2024-09-12T02:19:57.029 INFO:tasks.workunit.client.0.smithi003.stdout:config.status: creating config.h 2024-09-12T02:19:57.057 INFO:tasks.workunit.client.0.smithi003.stdout:make all-am 2024-09-12T02:19:57.062 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T02:19:57.063 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-12T02:19:57.239 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-12T02:19:57.243 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-12T02:19:57.331 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-12T02:19:57.334 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-12T02:19:57.422 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-12T02:19:57.425 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-12T02:19:57.513 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-12T02:19:57.516 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-12T02:19:57.640 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-12T02:19:57.643 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-12T02:19:57.691 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-12T02:19:57.694 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-12T02:19:57.736 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-12T02:19:57.738 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-12T02:19:57.784 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-12T02:19:57.787 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-12T02:19:57.991 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-12T02:19:57.994 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-12T02:19:58.126 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-12T02:19:58.129 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-12T02:19:58.288 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-12T02:19:58.291 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-12T02:19:58.361 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-12T02:19:58.364 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-12T02:19:58.444 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-12T02:19:58.447 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-12T02:19:58.550 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-12T02:19:58.553 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-12T02:19:58.583 INFO:tasks.workunit.client.0.smithi003.stderr:In file included from parser.h:22, 2024-09-12T02:19:58.583 INFO:tasks.workunit.client.0.smithi003.stderr: from parser.c:25: 2024-09-12T02:19:58.583 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘init_filesys’: 2024-09-12T02:19:58.583 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.583 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.584 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.584 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.584 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.584 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.584 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.585 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.585 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.585 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.586 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.587 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.587 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.587 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.587 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.587 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:19:58.588 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.589 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.590 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.591 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.592 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.593 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.594 INFO:tasks.workunit.client.0.smithi003.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.595 INFO:tasks.workunit.client.0.smithi003.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: | ^ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-12T02:19:58.596 INFO:tasks.workunit.client.0.smithi003.stderr: 36 | container_of(ptr, type, member) 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~ 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~ 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-12T02:19:58.597 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.629 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘get_optstr’: 2024-09-12T02:19:58.629 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-12T02:19:58.629 INFO:tasks.workunit.client.0.smithi003.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-12T02:19:58.630 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.630 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:200:14: note: source object declared here 2024-09-12T02:19:58.630 INFO:tasks.workunit.client.0.smithi003.stderr: 200 | char temp[BUFSIZE]; 2024-09-12T02:19:58.630 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~ 2024-09-12T02:19:58.636 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c: In function ‘size64_convert’: 2024-09-12T02:19:58.636 INFO:tasks.workunit.client.0.smithi003.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-12T02:19:58.636 INFO:tasks.workunit.client.0.smithi003.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-12T02:19:58.636 INFO:tasks.workunit.client.0.smithi003.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-12T02:19:58.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:58 smithi026 ceph-mon[35496]: pgmap v383: 97 pgs: 97 active+clean; 4.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 445 KiB/s wr, 128 op/s 2024-09-12T02:19:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:58 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:19:58.341+0000 7f6df7056640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T02:19:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:58 smithi003 ceph-mon[31476]: pgmap v383: 97 pgs: 97 active+clean; 4.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 445 KiB/s wr, 128 op/s 2024-09-12T02:19:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:58 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:58.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:58 smithi092 ceph-mon[36089]: pgmap v383: 97 pgs: 97 active+clean; 4.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 445 KiB/s wr, 128 op/s 2024-09-12T02:19:58.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:58 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:58.935 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-12T02:19:58.938 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-12T02:19:58.995 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-12T02:19:58.998 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-12T02:19:59.088 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-12T02:19:59.090 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-12T02:19:59.120 INFO:tasks.workunit.client.0.smithi003.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-12T02:19:59.124 INFO:tasks.workunit.client.0.smithi003.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-12T02:19:59.156 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-12T02:19:59.157 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd .. 2024-09-12T02:19:59.157 INFO:tasks.workunit.client.0.smithi003.stderr:+ mkdir tmp 2024-09-12T02:19:59.159 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd tmp 2024-09-12T02:19:59.159 INFO:tasks.workunit.client.0.smithi003.stderr:+ for f in $mydir/*.ffsb 2024-09-12T02:19:59.159 INFO:tasks.workunit.client.0.smithi003.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-12T02:19:59.160 INFO:tasks.workunit.client.0.smithi003.stdout:FFSB version 6.0-RC2 started 2024-09-12T02:19:59.160 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout:benchmark time = 300 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout:ThreadGroup 0 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout:================ 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: num_threads = 32 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: read_random = off 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: read_size = 0 (0B) 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: read_blocksize = 0 (0B) 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: read_skip = off 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: read_skipsize = 0 (0B) 2024-09-12T02:19:59.163 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: write_random = on 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: write_size = 5242880 (5MB) 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: fsync_file = 0 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: write_blocksize = 4096 (4KB) 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: wait time = 0 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: op weights 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: read = 0 (0.00%) 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: readall = 0 (0.00%) 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: write = 1 (100.00%) 2024-09-12T02:19:59.164 INFO:tasks.workunit.client.0.smithi003.stdout: create = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: append = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: delete = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: metaop = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: createdir = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: stat = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: writeall = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: writeall_fsync = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: open_close = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: write_fsync = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: create_fsync = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: append_fsync = 0 (0.00%) 2024-09-12T02:19:59.165 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout:FileSystem . 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout:========== 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: num_dirs = 0 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: starting files = 128 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: min file size = 104857600 (100MB) 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: max file size = 104857600 (100MB) 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: directio = off 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: alignedio = on 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: bufferedio = off 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.166 INFO:tasks.workunit.client.0.smithi003.stdout: aging is off 2024-09-12T02:19:59.167 INFO:tasks.workunit.client.0.smithi003.stdout: current utilization = 0.00% 2024-09-12T02:19:59.167 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:19:59.167 INFO:tasks.workunit.client.0.smithi003.stdout:checking existing fs: . 2024-09-12T02:19:59.167 INFO:tasks.workunit.client.0.smithi003.stdout:recreating new fileset 2024-09-12T02:19:59.167 INFO:tasks.workunit.client.0.smithi003.stderr:opendir: No such file or directory 2024-09-12T02:19:59.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:59 smithi026 ceph-mon[35496]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T02:19:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:59 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:19:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:19:59 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:59.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:59 smithi003 ceph-mon[31476]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T02:19:59.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:59 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:19:59.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:19:59 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:19:59.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:59 smithi092 ceph-mon[36089]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-12T02:19:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:59 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] up:active 2024-09-12T02:19:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:19:59 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:20:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:19:59.999+0000 7f6df7056640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:20:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:19:59.999+0000 7f6df7056640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T02:20:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:19:59.999+0000 7f6df7056640 -1 log_channel(cluster) log [ERR] : mds.f(mds.2): Metadata damage detected 2024-09-12T02:20:00.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:00 smithi026 ceph-mon[35496]: pgmap v384: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 529 KiB/s wr, 140 op/s 2024-09-12T02:20:00.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:00 smithi026 ceph-mon[35496]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:20:00.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:00 smithi026 ceph-mon[35496]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T02:20:00.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:00 smithi026 ceph-mon[35496]: mds.f(mds.2): Metadata damage detected 2024-09-12T02:20:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-mon[31476]: pgmap v384: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 529 KiB/s wr, 140 op/s 2024-09-12T02:20:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-mon[31476]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:20:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-mon[31476]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T02:20:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:00 smithi003 ceph-mon[31476]: mds.f(mds.2): Metadata damage detected 2024-09-12T02:20:00.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:00 smithi092 ceph-mon[36089]: pgmap v384: 97 pgs: 97 active+clean; 5.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 529 KiB/s wr, 140 op/s 2024-09-12T02:20:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:00 smithi092 ceph-mon[36089]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-09-12T02:20:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:00 smithi092 ceph-mon[36089]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-09-12T02:20:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:00 smithi092 ceph-mon[36089]: mds.f(mds.2): Metadata damage detected 2024-09-12T02:20:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.494+0000 7fa00c7fd640 1 -- 172.21.15.3:0/1208902039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 msgr2=0x7fa004108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.494+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/1208902039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa004108ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009ca0 tx=0x7f9ffc02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 -- 172.21.15.3:0/1208902039 shutdown_connections 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/1208902039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa004108ea0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/1208902039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa004101c50 0x7fa0041020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/1208902039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004100a60 0x7fa004100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 -- 172.21.15.3:0/1208902039 >> 172.21.15.3:0/1208902039 conn(0x7fa0040fc1c0 msgr2=0x7fa0040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 -- 172.21.15.3:0/1208902039 shutdown_connections 2024-09-12T02:20:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.495+0000 7fa00c7fd640 1 -- 172.21.15.3:0/1208902039 wait complete. 2024-09-12T02:20:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 Processor -- start 2024-09-12T02:20:01.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 -- start start 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa004100a60 0x7fa0041a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004101c50 0x7fa0041a3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa00410c800 con 0x7fa004102610 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa00410c680 con 0x7fa004101c50 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.496+0000 7fa00c7fd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa00410c980 con 0x7fa004100a60 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60068/0 (socket says 172.21.15.3:60068) 2024-09-12T02:20:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 -- 172.21.15.3:0/2775184709 learned_addr learned my addr 172.21.15.3:0/2775184709 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa009d98640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004101c50 0x7fa0041a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 -- 172.21.15.3:0/2775184709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa004100a60 msgr2=0x7fa0041a0cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa004100a60 0x7fa0041a0cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 -- 172.21.15.3:0/2775184709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004101c50 msgr2=0x7fa0041a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004101c50 0x7fa0041a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.497+0000 7fa00ad9a640 1 -- 172.21.15.3:0/2775184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0041a6200 con 0x7fa004102610 2024-09-12T02:20:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.498+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc003c10 tx=0x7f9ffc003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.498+0000 7f9ffb7fe640 1 -- 172.21.15.3:0/2775184709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc003e60 con 0x7fa004102610 2024-09-12T02:20:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.498+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0041a6490 con 0x7fa004102610 2024-09-12T02:20:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.498+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0041a69f0 con 0x7fa004102610 2024-09-12T02:20:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.498+0000 7f9ffb7fe640 1 -- 172.21.15.3:0/2775184709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc004720 con 0x7fa004102610 2024-09-12T02:20:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.500+0000 7f9ffb7fe640 1 -- 172.21.15.3:0/2775184709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc030e60 con 0x7fa004102610 2024-09-12T02:20:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.503+0000 7f9ffb7fe640 1 -- 172.21.15.3:0/2775184709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9ffc004240 con 0x7fa004102610 2024-09-12T02:20:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.504+0000 7fa00f680740 1 Processor -- start 2024-09-12T02:20:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.504+0000 7f9ffb7fe640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 0x7f9ff00a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.505+0000 7fa009d98640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 0x7f9ff00a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.505+0000 7f9ffb7fe640 1 -- 172.21.15.3:0/2775184709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9ffc0f06f0 con 0x7fa004102610 2024-09-12T02:20:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.505+0000 7fa00f680740 1 -- start start 2024-09-12T02:20:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.505+0000 7fa00f680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00f680740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00f680740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00f680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b71fc7f2f0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00f680740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b71fc27270 con 0x55b71fd21370 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00f680740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b71fc7b2b0 con 0x55b71fd1c2e0 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa009d98640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 0x7f9ff00a4f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4006fd0 tx=0x7f9ff4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00a599640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00a599640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60080/0 (socket says 172.21.15.3:60080) 2024-09-12T02:20:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.506+0000 7fa00a599640 1 -- 172.21.15.3:0/1483659082 learned_addr learned my addr 172.21.15.3:0/1483659082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 -- 172.21.15.3:0/1483659082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 msgr2=0x55b71fd20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 -- 172.21.15.3:0/1483659082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 msgr2=0x55b71fd23830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 -- 172.21.15.3:0/1483659082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71fc26340 con 0x55b71fd1b9a0 2024-09-12T02:20:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.507+0000 7fa00a599640 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa000009770 tx=0x7fa000009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45963e7dff69d866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.508+0000 7f9ff8ff9640 1 -- 172.21.15.3:0/1483659082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000015020 con 0x55b71fd1b9a0 2024-09-12T02:20:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.508+0000 7f9ff8ff9640 1 -- 172.21.15.3:0/1483659082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000011420 con 0x55b71fd1b9a0 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.508+0000 7f9ff8ff9640 1 -- 172.21.15.3:0/1483659082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0000105e0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.509+0000 7fa00f680740 1 -- 172.21.15.3:0/1483659082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 msgr2=0x55b71fd1bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.509+0000 7fa00f680740 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa000009770 tx=0x7fa000009c30 comp rx=0 tx=0).stop 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 -- 172.21.15.3:0/1483659082 shutdown_connections 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 --2- 172.21.15.3:0/1483659082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd1bda0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 -- 172.21.15.3:0/1483659082 >> 172.21.15.3:0/1483659082 conn(0x55b71fd17010 msgr2=0x55b71fd17410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 -- 172.21.15.3:0/1483659082 shutdown_connections 2024-09-12T02:20:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.510+0000 7fa00f680740 1 -- 172.21.15.3:0/1483659082 wait complete. 2024-09-12T02:20:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.511+0000 7fa00f680740 1 -- start start 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.511+0000 7fa00f680740 1 Processor -- start 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd25ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b71fd243e0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b71fc08290 con 0x55b71fd21370 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b71fd24560 con 0x55b71fd1c2e0 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa00f680740 10 client.0 _ll_register_callbacks cb 0x55b71fc39140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.512+0000 7fa009d98640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:44834/0 (socket says 172.21.15.3:44834) 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 -- 172.21.15.3:0/3039508109 learned_addr learned my addr 172.21.15.3:0/3039508109 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00a599640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd25ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 -- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 msgr2=0x55b71fd261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd261e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 -- 172.21.15.3:0/3039508109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 msgr2=0x55b71fd25ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd25ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.513+0000 7fa00ad9a640 1 -- 172.21.15.3:0/3039508109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71f992210 con 0x55b71fd21370 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.514+0000 7fa00a599640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd25ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.514+0000 7fa00ad9a640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0b98b0 tx=0x7f9ffc0b9240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9917366ca191eb16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.514+0000 7f9fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.515+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0bd5e0 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.515+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa000011420 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.515+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9ffc0b9550 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.515+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ffc0bcee0 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.516+0000 7fa00f680740 10 client.25535 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.516+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b71fa5e620 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.516+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc0d8020 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f9ffc08c400 con 0x55b71fd21370 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 10 client.25535 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 10 client.25535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 20 client.25535 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fd429a0 0x55b71fd44dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 4 client.25535 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.518+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b71fc8ca20 con 0x55b71fd429a0 2024-09-12T02:20:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.519+0000 7fa00a599640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fd429a0 0x55b71fd44dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.523+0000 7fa00a599640 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fd429a0 0x55b71fd44dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d66ba4bf11ea51 server_cookie=a37794627181bf31 in_seq=0 out_seq=0 2024-09-12T02:20:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.523+0000 7f9fedffb640 10 client.25535 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.525+0000 7f9fedffb640 1 -- 172.21.15.3:0/3039508109 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b71fc8ca20 con 0x55b71fd429a0 2024-09-12T02:20:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.525+0000 7f9fedffb640 10 client.25535 handle_command_reply: tid=0 2024-09-12T02:20:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 client.25535 shutdown 2024-09-12T02:20:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 msgr2=0x55b71fd2d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0b98b0 tx=0x7f9ffc0b9240 comp rx=0 tx=0).stop 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 shutdown_connections 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fd429a0 0x55b71fd44dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fd2d220 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 --2- 172.21.15.3:0/3039508109 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd25ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 >> 172.21.15.3:0/3039508109 conn(0x55b71fd17010 msgr2=0x55b71fd18820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7f9fedffb640 0 client.25535 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 shutdown_connections 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.526+0000 7fa00f680740 1 -- 172.21.15.3:0/3039508109 wait complete. 2024-09-12T02:20:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.527+0000 7fa00f680740 20 client.25535 trim_cache size 0 max 16384 2024-09-12T02:20:01.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.568+0000 7fa00f680740 1 Processor -- start 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.568+0000 7fa00f680740 1 -- start start 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd1b9a0 0x55b71fd20ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd21370 0x55b71fd25c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe800e480 con 0x55b71fd1c2e0 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80100b0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa00f680740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe800eb50 con 0x55b71fd21370 2024-09-12T02:20:01.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa009d98640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa009d98640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60112/0 (socket says 172.21.15.3:60112) 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.569+0000 7fa009d98640 1 -- 172.21.15.3:0/3706822894 learned_addr learned my addr 172.21.15.3:0/3706822894 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.570+0000 7fa00a599640 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd1b9a0 0x55b71fd20ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.571+0000 7fa009d98640 1 -- 172.21.15.3:0/3706822894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd21370 msgr2=0x55b71fd25c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.571+0000 7fa009d98640 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd21370 0x55b71fd25c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.571+0000 7fa009d98640 1 -- 172.21.15.3:0/3706822894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd1b9a0 msgr2=0x55b71fd20ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.571+0000 7fa009d98640 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd1b9a0 0x55b71fd20ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.571+0000 7fa009d98640 1 -- 172.21.15.3:0/3706822894 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b71fd20760 con 0x55b71fd1c2e0 2024-09-12T02:20:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.572+0000 7fa009d98640 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d8e0 tx=0x7f9ff4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b250be7cde7f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.573+0000 7f9feffff640 1 -- 172.21.15.3:0/3706822894 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400de60 con 0x55b71fd1c2e0 2024-09-12T02:20:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.573+0000 7f9feffff640 1 -- 172.21.15.3:0/3706822894 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ff4025070 con 0x55b71fd1c2e0 2024-09-12T02:20:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.574+0000 7f9feffff640 1 -- 172.21.15.3:0/3706822894 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40123e0 con 0x55b71fd1c2e0 2024-09-12T02:20:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.575+0000 7fa00f680740 1 -- 172.21.15.3:0/3706822894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 msgr2=0x55b71fd2d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.575+0000 7fa00f680740 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d8e0 tx=0x7f9ff4010000 comp rx=0 tx=0).stop 2024-09-12T02:20:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.576+0000 7fa00f680740 1 -- 172.21.15.3:0/3706822894 shutdown_connections 2024-09-12T02:20:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.576+0000 7fa00f680740 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd21370 0x55b71fd25c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.576+0000 7fa00f680740 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2d1e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.576+0000 7fa00f680740 1 --2- 172.21.15.3:0/3706822894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd1b9a0 0x55b71fd20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.576+0000 7fa00f680740 1 -- 172.21.15.3:0/3706822894 >> 172.21.15.3:0/3706822894 conn(0x55b71fd17010 msgr2=0x55b71fd1d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.577+0000 7fa00f680740 1 -- 172.21.15.3:0/3706822894 shutdown_connections 2024-09-12T02:20:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.577+0000 7fa00f680740 1 -- 172.21.15.3:0/3706822894 wait complete. 2024-09-12T02:20:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.578+0000 7fa00f680740 1 -- start start 2024-09-12T02:20:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.578+0000 7fa00f680740 1 Processor -- start 2024-09-12T02:20:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.580+0000 7fa00f680740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.581+0000 7fa00f680740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.581+0000 7fa00f680740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fdab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.582+0000 7fa00f680740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe800f950 con 0x55b71fd1b9a0 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.582+0000 7fa00f680740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe800f280 con 0x55b71fd21370 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.582+0000 7fa00f680740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe800c460 con 0x55b71fd1c2e0 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.582+0000 7fa00f680740 10 client.0 _ll_register_callbacks cb 0x7f9ffc0048d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.583+0000 7fa00a599640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.583+0000 7fa00a599640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60118/0 (socket says 172.21.15.3:60118) 2024-09-12T02:20:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.583+0000 7fa00a599640 1 -- 172.21.15.3:0/3976191065 learned_addr learned my addr 172.21.15.3:0/3976191065 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.585+0000 7fa009d98640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.586+0000 7fa00a599640 1 -- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 msgr2=0x55b71fd2a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.586+0000 7fa00a599640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.586+0000 7fa00a599640 1 -- 172.21.15.3:0/3976191065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 msgr2=0x55b71fdab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.586+0000 7fa00a599640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fdab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.586+0000 7fa00a599640 1 -- 172.21.15.3:0/3976191065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b71fd2ab30 con 0x55b71fd1b9a0 2024-09-12T02:20:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.587+0000 7fa00a599640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa00000b4b0 tx=0x7fa000011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5508146a36c9c41b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.587+0000 7f9fedffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.588+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000015040 con 0x55b71fd1b9a0 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.588+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa00000b8d0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.588+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff4025070 con 0x55b71fd1b9a0 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.589+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa00000bad0 con 0x55b71fd1b9a0 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.589+0000 7fa00f680740 10 client.16032 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.589+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b71fdacd60 con 0x55b71fd1b9a0 2024-09-12T02:20:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.589+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa00000bc70 con 0x55b71fd1b9a0 2024-09-12T02:20:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fa000023070 con 0x55b71fd1b9a0 2024-09-12T02:20:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 10 client.16032 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 20 client.16032 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fdad5b0 0x55b71fdaf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 4 client.16032 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:20:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.590+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9ff4025070 con 0x55b71fdad5b0 2024-09-12T02:20:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.593+0000 7fa009d98640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fdad5b0 0x55b71fdaf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.597+0000 7fa009d98640 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fdad5b0 0x55b71fdaf9d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8144a6377ee8c2af server_cookie=52e47e1a62955ec9 in_seq=0 out_seq=0 2024-09-12T02:20:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.597+0000 7f9fedffb640 10 client.16032 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.678+0000 7f9fedffb640 1 -- 172.21.15.3:0/3976191065 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9ff4025070 con 0x55b71fdad5b0 2024-09-12T02:20:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.678+0000 7f9fedffb640 10 client.16032 handle_command_reply: tid=0 2024-09-12T02:20:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.679+0000 7fa00f680740 1 client.16032 shutdown 2024-09-12T02:20:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.679+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 msgr2=0x55b71fd29f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.679+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa00000b4b0 tx=0x7fa000011f90 comp rx=0 tx=0).stop 2024-09-12T02:20:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.679+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 shutdown_connections 2024-09-12T02:20:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b71fdad5b0 0x55b71fdaf9d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b71fd21370 0x55b71fdab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b71fd1c2e0 0x55b71fd2a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 --2- 172.21.15.3:0/3976191065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b71fd1b9a0 0x55b71fd29f50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 >> 172.21.15.3:0/3976191065 conn(0x55b71fd17010 msgr2=0x55b71fd25c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7f9fedffb640 0 client.16032 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 shutdown_connections 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 1 -- 172.21.15.3:0/3976191065 wait complete. 2024-09-12T02:20:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.680+0000 7fa00f680740 20 client.16032 trim_cache size 0 max 16384 2024-09-12T02:20:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:20:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:20:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:20:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:20:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 msgr2=0x7f9ff00a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 0x7f9ff00a4f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4006fd0 tx=0x7f9ff4008040 comp rx=0 tx=0).stop 2024-09-12T02:20:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 msgr2=0x7fa0041a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc003c10 tx=0x7f9ffc003c40 comp rx=0 tx=0).stop 2024-09-12T02:20:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 shutdown_connections 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9ff00a2ad0 0x7f9ff00a4f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa004102610 0x7fa0041a5940 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004101c50 0x7fa0041a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 --2- 172.21.15.3:0/2775184709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa004100a60 0x7fa0041a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.682+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 >> 172.21.15.3:0/2775184709 conn(0x7fa0040fc1c0 msgr2=0x7fa00406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.683+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 shutdown_connections 2024-09-12T02:20:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:01.683+0000 7fa00c7fd640 1 -- 172.21.15.3:0/2775184709 wait complete. 2024-09-12T02:20:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:02 smithi026 ceph-mon[35496]: pgmap v385: 97 pgs: 97 active+clean; 6.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 571 KiB/s wr, 138 op/s 2024-09-12T02:20:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:02 smithi003 ceph-mon[31476]: pgmap v385: 97 pgs: 97 active+clean; 6.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 571 KiB/s wr, 138 op/s 2024-09-12T02:20:02.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:02 smithi092 ceph-mon[36089]: pgmap v385: 97 pgs: 97 active+clean; 6.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 571 KiB/s wr, 138 op/s 2024-09-12T02:20:04.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:04 smithi026 ceph-mon[35496]: pgmap v386: 97 pgs: 97 active+clean; 192 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 16 MiB/s wr, 144 op/s 2024-09-12T02:20:04.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:04 smithi003 ceph-mon[31476]: pgmap v386: 97 pgs: 97 active+clean; 192 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 16 MiB/s wr, 144 op/s 2024-09-12T02:20:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:04 smithi092 ceph-mon[36089]: pgmap v386: 97 pgs: 97 active+clean; 192 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 16 MiB/s wr, 144 op/s 2024-09-12T02:20:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:06 smithi026 ceph-mon[35496]: pgmap v387: 97 pgs: 97 active+clean; 786 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 65 MiB/s wr, 185 op/s 2024-09-12T02:20:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:06 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:20:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:06 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:20:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:06 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:20:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:06 smithi026 ceph-mon[35496]: scrub summary: idle 2024-09-12T02:20:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:06 smithi003 ceph-mon[31476]: pgmap v387: 97 pgs: 97 active+clean; 786 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 65 MiB/s wr, 185 op/s 2024-09-12T02:20:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:06 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:20:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:06 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:20:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:06 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:20:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:06 smithi003 ceph-mon[31476]: scrub summary: idle 2024-09-12T02:20:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:06 smithi092 ceph-mon[36089]: pgmap v387: 97 pgs: 97 active+clean; 786 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 65 MiB/s wr, 185 op/s 2024-09-12T02:20:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:06 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:20:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:06 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:20:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:06 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:20:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:06 smithi092 ceph-mon[36089]: scrub summary: idle 2024-09-12T02:20:08.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:08 smithi026 ceph-mon[35496]: pgmap v388: 97 pgs: 97 active+clean; 3.3 GiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 282 MiB/s wr, 240 op/s 2024-09-12T02:20:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:08 smithi003 ceph-mon[31476]: pgmap v388: 97 pgs: 97 active+clean; 3.3 GiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 282 MiB/s wr, 240 op/s 2024-09-12T02:20:08.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:08 smithi092 ceph-mon[36089]: pgmap v388: 97 pgs: 97 active+clean; 3.3 GiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 282 MiB/s wr, 240 op/s 2024-09-12T02:20:10.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:10 smithi026 ceph-mon[35496]: pgmap v389: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 396 MiB/s wr, 316 op/s 2024-09-12T02:20:10.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:10 smithi003 ceph-mon[31476]: pgmap v389: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 396 MiB/s wr, 316 op/s 2024-09-12T02:20:10.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:10 smithi092 ceph-mon[36089]: pgmap v389: 97 pgs: 97 active+clean; 4.6 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 396 MiB/s wr, 316 op/s 2024-09-12T02:20:12.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:12 smithi026 ceph-mon[35496]: pgmap v390: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 497 MiB/s wr, 329 op/s 2024-09-12T02:20:12.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:12 smithi003 ceph-mon[31476]: pgmap v390: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 497 MiB/s wr, 329 op/s 2024-09-12T02:20:12.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:12 smithi092 ceph-mon[36089]: pgmap v390: 97 pgs: 97 active+clean; 5.8 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 497 MiB/s wr, 329 op/s 2024-09-12T02:20:12.934 INFO:tasks.workunit.client.0.smithi003.stdout:fs setup took 13 secs 2024-09-12T02:20:14.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:14 smithi026 ceph-mon[35496]: pgmap v391: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 649 MiB/s wr, 398 op/s 2024-09-12T02:20:14.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:14 smithi003 ceph-mon[31476]: pgmap v391: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 649 MiB/s wr, 398 op/s 2024-09-12T02:20:14.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:14 smithi092 ceph-mon[36089]: pgmap v391: 97 pgs: 97 active+clean; 7.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 649 MiB/s wr, 398 op/s 2024-09-12T02:20:14.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 -- 172.21.15.3:0/3380324840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 msgr2=0x7fb2cc108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 --2- 172.21.15.3:0/3380324840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc108e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009ca0 tx=0x7fb2c402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 -- 172.21.15.3:0/3380324840 shutdown_connections 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 --2- 172.21.15.3:0/3380324840 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc108e80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 --2- 172.21.15.3:0/3380324840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2cc101c30 0x7fb2cc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 --2- 172.21.15.3:0/3380324840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc100a40 0x7fb2cc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 -- 172.21.15.3:0/3380324840 >> 172.21.15.3:0/3380324840 conn(0x7fb2cc0fc1c0 msgr2=0x7fb2cc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 -- 172.21.15.3:0/3380324840 shutdown_connections 2024-09-12T02:20:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.883+0000 7fb2d3404640 1 -- 172.21.15.3:0/3380324840 wait complete. 2024-09-12T02:20:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.884+0000 7fb2d3404640 1 Processor -- start 2024-09-12T02:20:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 -- start start 2024-09-12T02:20:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc100a40 0x7fb2cc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2cc101c30 0x7fb2cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10d840 con 0x7fb2cc1025f0 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10d6c0 con 0x7fb2cc101c30 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.885+0000 7fb2d3404640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc10d9c0 con 0x7fb2cc100a40 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55926/0 (socket says 172.21.15.3:55926) 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 -- 172.21.15.3:0/1160533666 learned_addr learned my addr 172.21.15.3:0/1160533666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 -- 172.21.15.3:0/1160533666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc100a40 msgr2=0x7fb2cc197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc100a40 0x7fb2cc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 -- 172.21.15.3:0/1160533666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2cc101c30 msgr2=0x7fb2cc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2cc101c30 0x7fb2cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.886+0000 7fb2d19a1640 1 -- 172.21.15.3:0/1160533666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc19fae0 con 0x7fb2cc1025f0 2024-09-12T02:20:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2d19a1640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009dc0 tx=0x7fb2c4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2c27fc640 1 -- 172.21.15.3:0/1160533666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4004240 con 0x7fb2cc1025f0 2024-09-12T02:20:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc19fd70 con 0x7fb2cc1025f0 2024-09-12T02:20:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2c27fc640 1 -- 172.21.15.3:0/1160533666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c4003e40 con 0x7fb2cc1025f0 2024-09-12T02:20:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2c27fc640 1 -- 172.21.15.3:0/1160533666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4004240 con 0x7fb2cc1025f0 2024-09-12T02:20:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.887+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc1a02c0 con 0x7fb2cc1025f0 2024-09-12T02:20:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.892+0000 7fb2c27fc640 1 -- 172.21.15.3:0/1160533666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb2c40306e0 con 0x7fb2cc1025f0 2024-09-12T02:20:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.892+0000 7fb2d6278740 1 Processor -- start 2024-09-12T02:20:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 -- start start 2024-09-12T02:20:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cacda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c439c0b2f0 con 0x55c439cad370 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c439bb3270 con 0x55c439ca82e0 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.893+0000 7fb2d6278740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c439c072b0 con 0x55c439ca79a0 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55938/0 (socket says 172.21.15.3:55938) 2024-09-12T02:20:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 -- 172.21.15.3:0/3277946319 learned_addr learned my addr 172.21.15.3:0/3277946319 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2c27fc640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 0x7fb2b80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 -- 172.21.15.3:0/3277946319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 msgr2=0x55c439ca7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 -- 172.21.15.3:0/3277946319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 msgr2=0x55c439cacda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cacda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2d099f640 1 -- 172.21.15.3:0/3277946319 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c439bb2340 con 0x55c439cad370 2024-09-12T02:20:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.894+0000 7fb2c27fc640 1 -- 172.21.15.3:0/1160533666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2c40f0300 con 0x7fb2cc1025f0 2024-09-12T02:20:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.895+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80017d0 tx=0x7fb2c8001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd6d4b09af289f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.895+0000 7fb2d19a1640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 0x7fb2b80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.896+0000 7fb2b77fe640 1 -- 172.21.15.3:0/3277946319 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800eaf0 con 0x55c439cad370 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.896+0000 7fb2b77fe640 1 -- 172.21.15.3:0/3277946319 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c800ec90 con 0x55c439cad370 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.896+0000 7fb2b77fe640 1 -- 172.21.15.3:0/3277946319 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c801b8c0 con 0x55c439cad370 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.897+0000 7fb2d6278740 1 -- 172.21.15.3:0/3277946319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 msgr2=0x55c439caf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.897+0000 7fb2d19a1640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 0x7fb2b80a5240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb2c402f350 tx=0x7fb2c4030a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.897+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80017d0 tx=0x7fb2c8001c90 comp rx=0 tx=0).stop 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 -- 172.21.15.3:0/3277946319 shutdown_connections 2024-09-12T02:20:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439caf830 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cacda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3277946319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 -- 172.21.15.3:0/3277946319 >> 172.21.15.3:0/3277946319 conn(0x55c439ca3010 msgr2=0x55c439ca3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 -- 172.21.15.3:0/3277946319 shutdown_connections 2024-09-12T02:20:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.898+0000 7fb2d6278740 1 -- 172.21.15.3:0/3277946319 wait complete. 2024-09-12T02:20:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.899+0000 7fb2d6278740 1 -- start start 2024-09-12T02:20:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.899+0000 7fb2d6278740 1 Processor -- start 2024-09-12T02:20:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55954/0 (socket says 172.21.15.3:55954) 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d099f640 1 -- 172.21.15.3:0/1038516399 learned_addr learned my addr 172.21.15.3:0/1038516399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439cb9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55c439cb0340 con 0x55c439ca79a0 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55c439b94290 con 0x55c439ca82e0 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.900+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c439cb04c0 con 0x55c439cad370 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.901+0000 7fb2d6278740 10 client.0 _ll_register_callbacks cb 0x55c439bc5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 -- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 msgr2=0x55c439cb9230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:14.903 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439cb9230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 -- 172.21.15.3:0/1038516399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 msgr2=0x55c439cb21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c43991e210 con 0x55c439ca79a0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2d099f640 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000ea0 tx=0x7fb2c8013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d31451606aed90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.902+0000 7fb2b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8001440 con 0x55c439ca79a0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2c8004020 con 0x55c439ca79a0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c800ec90 con 0x55c439ca79a0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2c8004220 con 0x55c439ca79a0 2024-09-12T02:20:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2d6278740 10 client.16044 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.903+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c4399ea620 con 0x55c439ca79a0 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c800ec90 con 0x55c439ca79a0 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fb2c802e050 con 0x55c439ca79a0 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 10 client.16044 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 20 client.16044 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439cbd840 0x55c439cbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 4 client.16044 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.904+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c439c18a20 con 0x55c439cbd840 2024-09-12T02:20:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.906+0000 7fb2d11a0640 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439cbd840 0x55c439cbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.909+0000 7fb2b4ff9640 10 client.16044 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.909+0000 7fb2d11a0640 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439cbd840 0x55c439cbfca0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4132fc93effdc69c server_cookie=1d6bdd5933732834 in_seq=0 out_seq=0 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.910+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/1038516399 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c439c18a20 con 0x55c439cbd840 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.910+0000 7fb2b4ff9640 10 client.16044 handle_command_reply: tid=0 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.910+0000 7fb2d6278740 1 client.16044 shutdown 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 msgr2=0x55c439cb1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000ea0 tx=0x7fb2c8013b20 comp rx=0 tx=0).stop 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 shutdown_connections 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439cbd840 0x55c439cbfca0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439cb9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 --2- 172.21.15.3:0/1038516399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb1cb0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 >> 172.21.15.3:0/1038516399 conn(0x55c439ca3010 msgr2=0x55c439ca4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2b4ff9640 0 client.16044 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.911+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 shutdown_connections 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.912+0000 7fb2d6278740 1 -- 172.21.15.3:0/1038516399 wait complete. 2024-09-12T02:20:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.912+0000 7fb2d6278740 20 client.16044 trim_cache size 0 max 16384 2024-09-12T02:20:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.944+0000 7fb2d6278740 1 Processor -- start 2024-09-12T02:20:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.944+0000 7fb2d6278740 1 -- start start 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.944+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.944+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.945+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.945+0000 7fb2d6278740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a6f20 con 0x55c439cad370 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.945+0000 7fb2d6278740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a8b50 con 0x55c439ca82e0 2024-09-12T02:20:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.945+0000 7fb2d6278740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a75f0 con 0x55c439ca79a0 2024-09-12T02:20:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.946+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.946+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55966/0 (socket says 172.21.15.3:55966) 2024-09-12T02:20:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.946+0000 7fb2d099f640 1 -- 172.21.15.3:0/4208127652 learned_addr learned my addr 172.21.15.3:0/4208127652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.946+0000 7fb2d099f640 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 -- 172.21.15.3:0/4208127652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 msgr2=0x55c439ca19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 -- 172.21.15.3:0/4208127652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 msgr2=0x55c439cb91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 -- 172.21.15.3:0/4208127652 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c439cb2e30 con 0x55c439cad370 2024-09-12T02:20:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.947+0000 7fb2d099f640 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007fd0 tx=0x7fb2c8001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1de7cb28422cadc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.948+0000 7fb2b6ffd640 1 -- 172.21.15.3:0/4208127652 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8007a20 con 0x55c439cad370 2024-09-12T02:20:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.948+0000 7fb2b6ffd640 1 -- 172.21.15.3:0/4208127652 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8001440 con 0x55c439cad370 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.948+0000 7fb2b6ffd640 1 -- 172.21.15.3:0/4208127652 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003ed0 con 0x55c439cad370 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 -- 172.21.15.3:0/4208127652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 msgr2=0x55c439ca1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8007fd0 tx=0x7fb2c8001c90 comp rx=0 tx=0).stop 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 -- 172.21.15.3:0/4208127652 shutdown_connections 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439cad370 0x55c439ca1ee0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 --2- 172.21.15.3:0/4208127652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439ca79a0 0x55c439ca19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 -- 172.21.15.3:0/4208127652 >> 172.21.15.3:0/4208127652 conn(0x55c439ca3010 msgr2=0x55c439ca9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 -- 172.21.15.3:0/4208127652 shutdown_connections 2024-09-12T02:20:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.949+0000 7fb2d6278740 1 -- 172.21.15.3:0/4208127652 wait complete. 2024-09-12T02:20:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.950+0000 7fb2d6278740 1 -- start start 2024-09-12T02:20:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.950+0000 7fb2d6278740 1 Processor -- start 2024-09-12T02:20:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.951+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.951+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d6278740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439d26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d6278740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a83f0 con 0x55c439ca79a0 2024-09-12T02:20:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d6278740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a7d20 con 0x55c439ca82e0 2024-09-12T02:20:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d6278740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc1a4f00 con 0x55c439cad370 2024-09-12T02:20:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d6278740 10 client.0 _ll_register_callbacks cb 0x7fb2c80016a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d099f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55978/0 (socket says 172.21.15.3:55978) 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d099f640 1 -- 172.21.15.3:0/3656583267 learned_addr learned my addr 172.21.15.3:0/3656583267 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.952+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439d26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.953+0000 7fb2d099f640 1 -- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 msgr2=0x55c439d26530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.953+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439d26530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.953+0000 7fb2d099f640 1 -- 172.21.15.3:0/3656583267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 msgr2=0x55c439cb6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.953+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.953+0000 7fb2d099f640 1 -- 172.21.15.3:0/3656583267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c439d26a70 con 0x55c439ca79a0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.954+0000 7fb2d099f640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8003840 tx=0x7fb2c8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdf0a2e30b021c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.954+0000 7fb2b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.954+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8040060 con 0x55c439ca79a0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.955+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb2c801ba50 con 0x55c439ca79a0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.955+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c8001440 con 0x55c439ca79a0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.955+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb2c801bc50 con 0x55c439ca79a0 2024-09-12T02:20:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.955+0000 7fb2d6278740 10 client.16050 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.956+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c439d27bb0 con 0x55c439ca79a0 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.956+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8001440 con 0x55c439ca79a0 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fb2c8038040 con 0x55c439ca79a0 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 10 client.16050 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 20 client.16050 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439d28530 0x55c439d2a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 4 client.16050 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:20:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.957+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c439c12d70 con 0x55c439d28530 2024-09-12T02:20:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.959+0000 7fb2d11a0640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439d28530 0x55c439d2a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.964+0000 7fb2d11a0640 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439d28530 0x55c439d2a950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcd0b90de346370 server_cookie=7ab8373b3d852fd3 in_seq=0 out_seq=0 2024-09-12T02:20:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:14.964+0000 7fb2b4ff9640 10 client.16050 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.154+0000 7fb2b4ff9640 1 -- 172.21.15.3:0/3656583267 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c439c12d70 con 0x55c439d28530 2024-09-12T02:20:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.154+0000 7fb2b4ff9640 10 client.16050 handle_command_reply: tid=0 2024-09-12T02:20:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.154+0000 7fb2d6278740 1 client.16050 shutdown 2024-09-12T02:20:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.154+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 msgr2=0x55c439cb6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.154+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8003840 tx=0x7fb2c8009040 comp rx=0 tx=0).stop 2024-09-12T02:20:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 shutdown_connections 2024-09-12T02:20:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55c439d28530 0x55c439d2a950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c439cad370 0x55c439d26530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2b4ff9640 0 client.16050 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c439ca82e0 0x55c439cb6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 --2- 172.21.15.3:0/3656583267 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55c439ca79a0 0x55c439cb6020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 >> 172.21.15.3:0/3656583267 conn(0x55c439ca3010 msgr2=0x55c439ca3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.155+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 shutdown_connections 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.156+0000 7fb2d6278740 1 -- 172.21.15.3:0/3656583267 wait complete. 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.156+0000 7fb2d6278740 20 client.16050 trim_cache size 0 max 16384 2024-09-12T02:20:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 msgr2=0x7fb2b80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 0x7fb2b80a5240 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb2c402f350 tx=0x7fb2c4030a80 comp rx=0 tx=0).stop 2024-09-12T02:20:15.159 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 msgr2=0x7fb2cc19f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009dc0 tx=0x7fb2c4003c20 comp rx=0 tx=0).stop 2024-09-12T02:20:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 shutdown_connections 2024-09-12T02:20:15.160 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb2b80a2d80 0x7fb2b80a5240 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb2cc1025f0 0x7fb2cc19f470 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2cc101c30 0x7fb2cc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 --2- 172.21.15.3:0/1160533666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2cc100a40 0x7fb2cc197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 >> 172.21.15.3:0/1160533666 conn(0x7fb2cc0fc1c0 msgr2=0x7fb2cc104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 shutdown_connections 2024-09-12T02:20:15.161 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:15.158+0000 7fb2d3404640 1 -- 172.21.15.3:0/1160533666 wait complete. 2024-09-12T02:20:16.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:16 smithi026 ceph-mon[35496]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 736 MiB/s wr, 388 op/s 2024-09-12T02:20:16.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:16 smithi003 ceph-mon[31476]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 736 MiB/s wr, 388 op/s 2024-09-12T02:20:16.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:16 smithi092 ceph-mon[36089]: pgmap v392: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 736 MiB/s wr, 388 op/s 2024-09-12T02:20:16.873 INFO:tasks.workunit.client.0.smithi003.stdout:Syncing()...3 sec 2024-09-12T02:20:16.874 INFO:tasks.workunit.client.0.smithi003.stdout:Starting Actual Benchmark At: Thu Sep 12 02:20:16 2024 2024-09-12T02:20:16.874 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:20:18.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:18 smithi026 ceph-mon[35496]: pgmap v393: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 869 MiB/s wr, 387 op/s 2024-09-12T02:20:18.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:18 smithi003 ceph-mon[31476]: pgmap v393: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 869 MiB/s wr, 387 op/s 2024-09-12T02:20:18.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:18 smithi092 ceph-mon[36089]: pgmap v393: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 869 MiB/s wr, 387 op/s 2024-09-12T02:20:20.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:20 smithi026 ceph-mon[35496]: pgmap v394: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 732 MiB/s wr, 1.02k op/s 2024-09-12T02:20:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:20 smithi003 ceph-mon[31476]: pgmap v394: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 732 MiB/s wr, 1.02k op/s 2024-09-12T02:20:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:20 smithi092 ceph-mon[36089]: pgmap v394: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 732 MiB/s wr, 1.02k op/s 2024-09-12T02:20:22.630 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T02:20:22.630 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:20:22.689 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:22 smithi003 ceph-mon[31476]: pgmap v395: 97 pgs: 97 active+clean; 12 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 673 MiB/s wr, 2.93k op/s 2024-09-12T02:20:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:22 smithi026 ceph-mon[35496]: pgmap v395: 97 pgs: 97 active+clean; 12 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 673 MiB/s wr, 2.93k op/s 2024-09-12T02:20:22.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:22 smithi092 ceph-mon[36089]: pgmap v395: 97 pgs: 97 active+clean; 12 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 673 MiB/s wr, 2.93k op/s 2024-09-12T02:20:23.249 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:20:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:24 smithi026 ceph-mon[35496]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 624 MiB/s wr, 8.61k op/s 2024-09-12T02:20:24.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:24 smithi003 ceph-mon[31476]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 624 MiB/s wr, 8.61k op/s 2024-09-12T02:20:24.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:24 smithi092 ceph-mon[36089]: pgmap v396: 97 pgs: 97 active+clean; 13 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 624 MiB/s wr, 8.61k op/s 2024-09-12T02:20:25.720 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:20:26.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:26 smithi026 ceph-mon[35496]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 507 MiB/s wr, 13.16k op/s 2024-09-12T02:20:26.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:26 smithi003 ceph-mon[31476]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 507 MiB/s wr, 13.16k op/s 2024-09-12T02:20:26.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:26 smithi092 ceph-mon[36089]: pgmap v397: 97 pgs: 97 active+clean; 13 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 507 MiB/s wr, 13.16k op/s 2024-09-12T02:20:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.283+0000 7f47aaae7640 1 -- 172.21.15.3:0/535415768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09ce30 msgr2=0x7f479c09d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.283+0000 7f47aaae7640 1 --2- 172.21.15.3:0/535415768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09ce30 0x7f479c09d2b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f47a001a910 tx=0x7f47a003b430 comp rx=0 tx=0).stop 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.285+0000 7f47aaae7640 1 -- 172.21.15.3:0/535415768 shutdown_connections 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.285+0000 7f47aaae7640 1 --2- 172.21.15.3:0/535415768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09d7f0 0x7f479c0a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.285+0000 7f47aaae7640 1 --2- 172.21.15.3:0/535415768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09ce30 0x7f479c09d2b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.285+0000 7f47aaae7640 1 --2- 172.21.15.3:0/535415768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c09bc40 0x7f479c09c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.285+0000 7f47aaae7640 1 -- 172.21.15.3:0/535415768 >> 172.21.15.3:0/535415768 conn(0x7f479c00ae50 msgr2=0x7f479c00d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.289+0000 7f47aaae7640 1 -- 172.21.15.3:0/535415768 shutdown_connections 2024-09-12T02:20:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.293+0000 7f47aaae7640 1 -- 172.21.15.3:0/535415768 wait complete. 2024-09-12T02:20:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.294+0000 7f47aaae7640 1 Processor -- start 2024-09-12T02:20:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 -- start start 2024-09-12T02:20:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c09ce30 0x7f479c012ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09d7f0 0x7f479c142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0a7950 con 0x7f479c09d7f0 2024-09-12T02:20:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0a77d0 con 0x7f479c09ce30 2024-09-12T02:20:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47aaae7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f479c0a7ad0 con 0x7f479c09bc40 2024-09-12T02:20:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47a9ae5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47a9ae5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36198/0 (socket says 172.21.15.3:36198) 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.295+0000 7f47a9ae5640 1 -- 172.21.15.3:0/712723927 learned_addr learned my addr 172.21.15.3:0/712723927 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47aa2e6640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09d7f0 0x7f479c142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 -- 172.21.15.3:0/712723927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c09ce30 msgr2=0x7f479c012ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c09ce30 0x7f479c012ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 -- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09d7f0 msgr2=0x7f479c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09d7f0 0x7f479c142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 -- 172.21.15.3:0/712723927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1425a0 con 0x7f479c09bc40 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.296+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f479800d770 tx=0x7f479800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.297+0000 7f4796ffd640 1 -- 172.21.15.3:0/712723927 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798017070 con 0x7f479c09bc40 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.297+0000 7f4796ffd640 1 -- 172.21.15.3:0/712723927 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798009c30 con 0x7f479c09bc40 2024-09-12T02:20:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.297+0000 7f4796ffd640 1 -- 172.21.15.3:0/712723927 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798003dd0 con 0x7f479c09bc40 2024-09-12T02:20:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.300+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c142770 con 0x7f479c09bc40 2024-09-12T02:20:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.300+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c142c60 con 0x7f479c09bc40 2024-09-12T02:20:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.308+0000 7f47aebbc740 1 Processor -- start 2024-09-12T02:20:28.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.310+0000 7f47aebbc740 1 -- start start 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 0x5611884bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bc4b0 0x5611884c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56118841f2f0 con 0x5611884bc4b0 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611883c7270 con 0x5611884bbb70 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.311+0000 7f47aebbc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56118841b2b0 con 0x5611884c1540 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.312+0000 7f47a92e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.312+0000 7f47a92e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36214/0 (socket says 172.21.15.3:36214) 2024-09-12T02:20:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.312+0000 7f47a92e4640 1 -- 172.21.15.3:0/1171403922 learned_addr learned my addr 172.21.15.3:0/1171403922 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.313+0000 7f47aa2e6640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 0x5611884bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.313+0000 7f4796ffd640 1 -- 172.21.15.3:0/712723927 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4798004020 con 0x7f479c09bc40 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.314+0000 7f4796ffd640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 0x7f47900a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f4796ffd640 1 -- 172.21.15.3:0/712723927 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f47980ce9e0 con 0x7f479c09bc40 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bc4b0 0x5611884c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a92e4640 1 -- 172.21.15.3:0/1171403922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 msgr2=0x5611884bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a92e4640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 0x5611884bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a92e4640 1 -- 172.21.15.3:0/1171403922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bc4b0 msgr2=0x5611884c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a92e4640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bc4b0 0x5611884c0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.315+0000 7f47a92e4640 1 -- 172.21.15.3:0/1171403922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611883c6340 con 0x5611884c1540 2024-09-12T02:20:28.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.316+0000 7f47aa2e6640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 0x5611884bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:20:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.317+0000 7f47a92e4640 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47a0004e80 tx=0x7f47a0005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0fb65ba6deb99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.318+0000 7f47aa2e6640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 0x7f47900a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.318+0000 7f478ffff640 1 -- 172.21.15.3:0/1171403922 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0046030 con 0x5611884c1540 2024-09-12T02:20:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.318+0000 7f478ffff640 1 -- 172.21.15.3:0/1171403922 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a00049e0 con 0x5611884c1540 2024-09-12T02:20:28.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.318+0000 7f478ffff640 1 -- 172.21.15.3:0/1171403922 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00035f0 con 0x5611884c1540 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.319+0000 7f47aebbc740 1 -- 172.21.15.3:0/1171403922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 msgr2=0x5611884c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.319+0000 7f47aebbc740 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47a0004e80 tx=0x7f47a0005e80 comp rx=0 tx=0).stop 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.319+0000 7f47aa2e6640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 0x7f47900a5220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f47a4050d60 tx=0x7f47a4067f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.319+0000 7f47aebbc740 1 -- 172.21.15.3:0/1171403922 shutdown_connections 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.319+0000 7f47aebbc740 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884c1540 0x5611884c3a00 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47a0004e80 tx=0x7f47a0005e80 comp rx=0 tx=0).stop 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.320+0000 7f47aebbc740 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bc4b0 0x5611884c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.320+0000 7f47aebbc740 1 --2- 172.21.15.3:0/1171403922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bbb70 0x5611884bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.320+0000 7f47aebbc740 1 -- 172.21.15.3:0/1171403922 >> 172.21.15.3:0/1171403922 conn(0x5611884b71e0 msgr2=0x5611884b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.321+0000 7f47aebbc740 1 -- 172.21.15.3:0/1171403922 shutdown_connections 2024-09-12T02:20:28.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.321+0000 7f47aebbc740 1 -- 172.21.15.3:0/1171403922 wait complete. 2024-09-12T02:20:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.321+0000 7f47aebbc740 1 -- start start 2024-09-12T02:20:28.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.322+0000 7f47aebbc740 1 Processor -- start 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884cb500 0x5611884cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5611884c44b0 con 0x5611884bbb70 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5611883a8290 con 0x5611884bc4b0 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5611884c4630 con 0x5611884cb500 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.323+0000 7f47aebbc740 10 client.0 _ll_register_callbacks cb 0x5611883d9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.324+0000 7f47a9ae5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.324+0000 7f47a9ae5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33140/0 (socket says 172.21.15.3:33140) 2024-09-12T02:20:28.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.324+0000 7f47a9ae5640 1 -- 172.21.15.3:0/745783179 learned_addr learned my addr 172.21.15.3:0/745783179 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.325+0000 7f47a9ae5640 1 -- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884cb500 msgr2=0x5611884cd8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.325+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884cb500 0x5611884cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.325+0000 7f47a9ae5640 1 -- 172.21.15.3:0/745783179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 msgr2=0x5611884c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.325+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.325+0000 7f47a9ae5640 1 -- 172.21.15.3:0/745783179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561188132210 con 0x5611884bbb70 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.326+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f47980a3910 tx=0x7f479809ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a97232d06e834b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.326+0000 7f478d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.327+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798020070 con 0x5611884bbb70 2024-09-12T02:20:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.327+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a00049e0 con 0x5611884bbb70 2024-09-12T02:20:28.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.327+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f47980cc580 con 0x5611884bbb70 2024-09-12T02:20:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.328+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f479809f580 con 0x5611884bbb70 2024-09-12T02:20:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.328+0000 7f47aebbc740 10 client.16062 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.328+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5611881fe620 con 0x5611884bbb70 2024-09-12T02:20:28.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.328+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a00049e0 con 0x5611884bbb70 2024-09-12T02:20:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f47980bb0b0 con 0x5611884bbb70 2024-09-12T02:20:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 10 client.16062 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 20 client.16062 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5611884d2510 0x5611884d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 4 client.16062 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.329+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118842ca20 con 0x5611884d2510 2024-09-12T02:20:28.332 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.331+0000 7f47a92e4640 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5611884d2510 0x5611884d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.346+0000 7f47a92e4640 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5611884d2510 0x5611884d49d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8f70eb1242aabe server_cookie=a550be0e9a9b1274 in_seq=0 out_seq=0 2024-09-12T02:20:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.346+0000 7f478d7fa640 10 client.16062 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.349+0000 7f478d7fa640 1 -- 172.21.15.3:0/745783179 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56118842ca20 con 0x5611884d2510 2024-09-12T02:20:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.350+0000 7f478d7fa640 10 client.16062 handle_command_reply: tid=0 2024-09-12T02:20:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.350+0000 7f47aebbc740 1 client.16062 shutdown 2024-09-12T02:20:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.357+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 msgr2=0x5611884c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.357+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f47980a3910 tx=0x7f479809ab30 comp rx=0 tx=0).stop 2024-09-12T02:20:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 shutdown_connections 2024-09-12T02:20:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5611884d2510 0x5611884d49d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884cb500 0x5611884cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 --2- 172.21.15.3:0/745783179 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884bbb70 0x5611884c6000 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.358+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 >> 172.21.15.3:0/745783179 conn(0x5611884b71e0 msgr2=0x5611884b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.359+0000 7f478d7fa640 0 client.16062 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:28.360 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.359+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 shutdown_connections 2024-09-12T02:20:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.360+0000 7f47aebbc740 1 -- 172.21.15.3:0/745783179 wait complete. 2024-09-12T02:20:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.360+0000 7f47aebbc740 20 client.16062 trim_cache size 0 max 16384 2024-09-12T02:20:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.401+0000 7f47aebbc740 1 Processor -- start 2024-09-12T02:20:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.401+0000 7f47aebbc740 1 -- start start 2024-09-12T02:20:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.402+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.402+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x5611884c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.402+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 0x5611884bdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.402+0000 7f47aebbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f479c14baf0 con 0x5611884cb500 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47aebbc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c14d720 con 0x5611884bc4b0 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47aebbc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f479c14c1c0 con 0x5611884bd830 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47aa2e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 0x5611884bdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47aa2e6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 0x5611884bdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36234/0 (socket says 172.21.15.3:36234) 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47aa2e6640 1 -- 172.21.15.3:0/3077477117 learned_addr learned my addr 172.21.15.3:0/3077477117 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:28.405 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.403+0000 7f47a92e4640 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x5611884c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 -- 172.21.15.3:0/3077477117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 msgr2=0x5611884bdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 0x5611884bdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 -- 172.21.15.3:0/3077477117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 msgr2=0x5611884c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 -- 172.21.15.3:0/3077477117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611884c5b40 con 0x5611884cb500 2024-09-12T02:20:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.404+0000 7f47a92e4640 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x5611884c4f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f47a0046000 tx=0x7f47a003b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5867d164ae57365 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.406+0000 7f478f7fe640 1 -- 172.21.15.3:0/3077477117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0004c70 con 0x5611884cb500 2024-09-12T02:20:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.406+0000 7f478f7fe640 1 -- 172.21.15.3:0/3077477117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a0005850 con 0x5611884cb500 2024-09-12T02:20:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.406+0000 7f478f7fe640 1 -- 172.21.15.3:0/3077477117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00035f0 con 0x5611884cb500 2024-09-12T02:20:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.407+0000 7f47aebbc740 1 -- 172.21.15.3:0/3077477117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 msgr2=0x5611884c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.407+0000 7f47aebbc740 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x5611884c4f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f47a0046000 tx=0x7f47a003b430 comp rx=0 tx=0).stop 2024-09-12T02:20:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.412+0000 7f47aebbc740 1 -- 172.21.15.3:0/3077477117 shutdown_connections 2024-09-12T02:20:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.412+0000 7f47aebbc740 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bd830 0x5611884bdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.412+0000 7f47aebbc740 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x5611884c4f70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.412+0000 7f47aebbc740 1 --2- 172.21.15.3:0/3077477117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bc4b0 0x5611884c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.412+0000 7f47aebbc740 1 -- 172.21.15.3:0/3077477117 >> 172.21.15.3:0/3077477117 conn(0x5611884b71e0 msgr2=0x5611884b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.414+0000 7f47aebbc740 1 -- 172.21.15.3:0/3077477117 shutdown_connections 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.414+0000 7f47aebbc740 1 -- 172.21.15.3:0/3077477117 wait complete. 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.414+0000 7f47aebbc740 1 -- start start 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.414+0000 7f47aebbc740 1 Processor -- start 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bc4b0 0x5611884e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 0x5611884e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x56118853b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f479c14cfc0 con 0x5611884cb500 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f479c14c8f0 con 0x5611884bd830 2024-09-12T02:20:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f479c149ad0 con 0x5611884bc4b0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.415+0000 7f47aebbc740 10 client.0 _ll_register_callbacks cb 0x7f479809afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.416+0000 7f47aa2e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x56118853b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.416+0000 7f47aa2e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x56118853b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33164/0 (socket says 172.21.15.3:33164) 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.416+0000 7f47aa2e6640 1 -- 172.21.15.3:0/2517836487 learned_addr learned my addr 172.21.15.3:0/2517836487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.416+0000 7f47a92e4640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 0x5611884e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.416+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bc4b0 0x5611884e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.417+0000 7f47a92e4640 1 -- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bc4b0 msgr2=0x5611884e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.417+0000 7f47a92e4640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bc4b0 0x5611884e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.417+0000 7f47a92e4640 1 -- 172.21.15.3:0/2517836487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 msgr2=0x56118853b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.417+0000 7f47a92e4640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x56118853b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.417+0000 7f47a92e4640 1 -- 172.21.15.3:0/2517836487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611884d4d40 con 0x5611884bd830 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.418+0000 7f47a92e4640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 0x5611884e3c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47a001a860 tx=0x7f47a0000eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f8719de7f81b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.418+0000 7f478d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:20:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.420+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0063040 con 0x5611884bd830 2024-09-12T02:20:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.421+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a0005850 con 0x5611884bd830 2024-09-12T02:20:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.421+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f47a0005460 con 0x5611884bd830 2024-09-12T02:20:28.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.421+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47a00049e0 con 0x5611884bd830 2024-09-12T02:20:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.423+0000 7f47aebbc740 10 client.25547 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:28.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.423+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56118853c6a0 con 0x5611884bd830 2024-09-12T02:20:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.425+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a0005850 con 0x5611884bd830 2024-09-12T02:20:28.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f47a0066070 con 0x5611884bd830 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 10 client.25547 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 20 client.25547 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56118853cff0 0x56118853f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 4 client.25547 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561188426d70 con 0x56118853cff0 2024-09-12T02:20:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.426+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56118853cff0 0x56118853f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.430+0000 7f47a9ae5640 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56118853cff0 0x56118853f410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12dd3709a71c1a3f server_cookie=6d48d2e65afaa279 in_seq=0 out_seq=0 2024-09-12T02:20:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.430+0000 7f478d7fa640 10 client.25547 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:28.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:28 smithi026 ceph-mon[35496]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 440 MiB/s wr, 17.35k op/s 2024-09-12T02:20:28.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:28 smithi003 ceph-mon[31476]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 440 MiB/s wr, 17.35k op/s 2024-09-12T02:20:28.788 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:28 smithi092 ceph-mon[36089]: pgmap v398: 97 pgs: 97 active+clean; 13 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 440 MiB/s wr, 17.35k op/s 2024-09-12T02:20:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.844+0000 7f478d7fa640 1 -- 172.21.15.3:0/2517836487 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561188426d70 con 0x56118853cff0 2024-09-12T02:20:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.844+0000 7f478d7fa640 10 client.25547 handle_command_reply: tid=0 2024-09-12T02:20:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.844+0000 7f47aebbc740 1 client.25547 shutdown 2024-09-12T02:20:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.845+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 msgr2=0x5611884e3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.845+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 0x5611884e3c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f47a001a860 tx=0x7f47a0000eb0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 shutdown_connections 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56118853cff0 0x56118853f410 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5611884cb500 0x56118853b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611884bd830 0x5611884e3c80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 --2- 172.21.15.3:0/2517836487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5611884bc4b0 0x5611884e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 >> 172.21.15.3:0/2517836487 conn(0x5611884b71e0 msgr2=0x5611884c4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f478d7fa640 0 client.25547 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 shutdown_connections 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.846+0000 7f47aebbc740 1 -- 172.21.15.3:0/2517836487 wait complete. 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.847+0000 7f47aebbc740 20 client.25547 trim_cache size 0 max 16384 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:20:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:20:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:20:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:20:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 msgr2=0x7f47900a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 0x7f47900a5220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f47a4050d60 tx=0x7f47a4067f10 comp rx=0 tx=0).stop 2024-09-12T02:20:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 msgr2=0x7f479c012ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f479800d770 tx=0x7f479800dc30 comp rx=0 tx=0).stop 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 shutdown_connections 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47900a2d60 0x7f47900a5220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f479c09d7f0 0x7f479c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.852+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f479c09ce30 0x7f479c012ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.853+0000 7f47aaae7640 1 --2- 172.21.15.3:0/712723927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f479c09bc40 0x7f479c012ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.853+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 >> 172.21.15.3:0/712723927 conn(0x7f479c00ae50 msgr2=0x7f479c0a01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.853+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 shutdown_connections 2024-09-12T02:20:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:28.853+0000 7f47aaae7640 1 -- 172.21.15.3:0/712723927 wait complete. 2024-09-12T02:20:30.589 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:20:30.589 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"btime":"2024-09-12T02:19:58:341951+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:19:58.176754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.186+0000 7f5a7214e640 1 -- 172.21.15.3:0/4270878253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c10ae80 msgr2=0x7f5a6c113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.186+0000 7f5a7214e640 1 --2- 172.21.15.3:0/4270878253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c10ae80 0x7f5a6c113370 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009900 tx=0x7f5a6002a400 comp rx=0 tx=0).stop 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.187+0000 7f5a7214e640 1 -- 172.21.15.3:0/4270878253 shutdown_connections 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.187+0000 7f5a7214e640 1 --2- 172.21.15.3:0/4270878253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c113940 0x7f5a6c115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.187+0000 7f5a7214e640 1 --2- 172.21.15.3:0/4270878253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c10ae80 0x7f5a6c113370 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009900 tx=0x7f5a6002a400 comp rx=0 tx=0).stop 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.187+0000 7f5a7214e640 1 --2- 172.21.15.3:0/4270878253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c10a560 0x7f5a6c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.187+0000 7f5a7214e640 1 -- 172.21.15.3:0/4270878253 >> 172.21.15.3:0/4270878253 conn(0x7f5a6c06e140 msgr2=0x7f5a6c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.188+0000 7f5a7214e640 1 -- 172.21.15.3:0/4270878253 shutdown_connections 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.188+0000 7f5a7214e640 1 -- 172.21.15.3:0/4270878253 wait complete. 2024-09-12T02:20:30.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.189+0000 7f5a7214e640 1 Processor -- start 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.189+0000 7f5a7214e640 1 -- start start 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c10a560 0x7f5a6c1ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 0x7f5a6c1a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c119690 con 0x7f5a6c10a560 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c119510 con 0x7f5a6c113940 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a7214e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c119810 con 0x7f5a6c1a7370 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a6affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:44578/0 (socket says 172.21.15.3:44578) 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a6affd640 1 -- 172.21.15.3:0/219402170 learned_addr learned my addr 172.21.15.3:0/219402170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:30.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.190+0000 7f5a6bfff640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 0x7f5a6c1a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6b7fe640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c10a560 0x7f5a6c1ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6affd640 1 -- 172.21.15.3:0/219402170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 msgr2=0x7f5a6c1a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6affd640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 0x7f5a6c1a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6affd640 1 -- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c10a560 msgr2=0x7f5a6c1ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6affd640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c10a560 0x7f5a6c1ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.191+0000 7f5a6affd640 1 -- 172.21.15.3:0/219402170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c1a7f50 con 0x7f5a6c113940 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.192+0000 7f5a6bfff640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 0x7f5a6c1a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.194+0000 7f5a6affd640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002a9f0 tx=0x7f5a60003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.196+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6003f020 con 0x7f5a6c113940 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.196+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a60031c90 con 0x7f5a6c113940 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.196+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60035070 con 0x7f5a6c113940 2024-09-12T02:20:30.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.196+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c1b3ef0 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.196+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1b4290 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.200+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5a60042020 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.201+0000 7f5a68ff9640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 0x7f5a540a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.201+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5a600ef140 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.202+0000 7f5a6b7fe640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 0x7f5a540a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.203+0000 7f5a6b7fe640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 0x7f5a540a5250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005fd0 tx=0x7f5a64009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.214+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58005290 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.226+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a60049070 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.603+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a58005580 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.605+0000 7f5a68ff9640 1 -- 172.21.15.3:0/219402170 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11954 (secure 0 0 0) 0x7f5a60049070 con 0x7f5a6c113940 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 33 2024-09-12T02:20:30.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 msgr2=0x7f5a540a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 0x7f5a540a5250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5a64005fd0 tx=0x7f5a64009500 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 msgr2=0x7f5a6c1ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5a6002a9f0 tx=0x7f5a60003b90 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 shutdown_connections 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5a540a2d90 0x7f5a540a5250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a6c1a7370 0x7f5a6c1a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.609+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a6c113940 0x7f5a6c1ad5d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.610+0000 7f5a7214e640 1 --2- 172.21.15.3:0/219402170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5a6c10a560 0x7f5a6c1ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.610+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 >> 172.21.15.3:0/219402170 conn(0x7f5a6c06e140 msgr2=0x7f5a6c070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.610+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 shutdown_connections 2024-09-12T02:20:30.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:29.610+0000 7f5a7214e640 1 -- 172.21.15.3:0/219402170 wait complete. 2024-09-12T02:20:30.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:30 smithi026 ceph-mon[35496]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 295 MiB/s wr, 21.68k op/s 2024-09-12T02:20:30.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:30 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/219402170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:20:30.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:30 smithi003 ceph-mon[31476]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 295 MiB/s wr, 21.68k op/s 2024-09-12T02:20:30.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:30 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/219402170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:20:30.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:30 smithi092 ceph-mon[36089]: pgmap v399: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 295 MiB/s wr, 21.68k op/s 2024-09-12T02:20:30.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:30 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/219402170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:20:31.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:31 smithi026 ceph-mon[35496]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 248 MiB/s wr, 24.39k op/s 2024-09-12T02:20:31.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:31 smithi003 ceph-mon[31476]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 248 MiB/s wr, 24.39k op/s 2024-09-12T02:20:31.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:31 smithi092 ceph-mon[36089]: pgmap v400: 97 pgs: 97 active+clean; 13 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 248 MiB/s wr, 24.39k op/s 2024-09-12T02:20:33.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:33 smithi003 ceph-mon[31476]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 236 MiB/s wr, 27.15k op/s 2024-09-12T02:20:33.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:33 smithi092 ceph-mon[36089]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 236 MiB/s wr, 27.15k op/s 2024-09-12T02:20:34.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:33 smithi026 ceph-mon[35496]: pgmap v401: 97 pgs: 97 active+clean; 13 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 236 MiB/s wr, 27.15k op/s 2024-09-12T02:20:36.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:35 smithi026 ceph-mon[35496]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 231 MiB/s wr, 26.52k op/s 2024-09-12T02:20:36.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:35 smithi003 ceph-mon[31476]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 231 MiB/s wr, 26.52k op/s 2024-09-12T02:20:36.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:35 smithi092 ceph-mon[36089]: pgmap v402: 97 pgs: 97 active+clean; 13 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 231 MiB/s wr, 26.52k op/s 2024-09-12T02:20:38.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:37 smithi026 ceph-mon[35496]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 243 MiB/s wr, 26.77k op/s 2024-09-12T02:20:38.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:37 smithi003 ceph-mon[31476]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 243 MiB/s wr, 26.77k op/s 2024-09-12T02:20:38.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:37 smithi092 ceph-mon[36089]: pgmap v403: 97 pgs: 97 active+clean; 13 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 243 MiB/s wr, 26.77k op/s 2024-09-12T02:20:40.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:39 smithi026 ceph-mon[35496]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 250 MiB/s wr, 26.72k op/s 2024-09-12T02:20:40.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:39 smithi003 ceph-mon[31476]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 250 MiB/s wr, 26.72k op/s 2024-09-12T02:20:40.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:39 smithi092 ceph-mon[36089]: pgmap v404: 97 pgs: 97 active+clean; 13 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 250 MiB/s wr, 26.72k op/s 2024-09-12T02:20:41.458 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:20:41.498 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T02:20:41.498 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 -- 172.21.15.3:0/2318955028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec98101c20 msgr2=0x7fec981020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 --2- 172.21.15.3:0/2318955028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec98101c20 0x7fec981020a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fec88009900 tx=0x7fec8802a410 comp rx=0 tx=0).stop 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 -- 172.21.15.3:0/2318955028 shutdown_connections 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 --2- 172.21.15.3:0/2318955028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec981025e0 0x7fec98108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 --2- 172.21.15.3:0/2318955028 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec98101c20 0x7fec981020a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 --2- 172.21.15.3:0/2318955028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 -- 172.21.15.3:0/2318955028 >> 172.21.15.3:0/2318955028 conn(0x7fec980fc190 msgr2=0x7fec980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 -- 172.21.15.3:0/2318955028 shutdown_connections 2024-09-12T02:20:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 -- 172.21.15.3:0/2318955028 wait complete. 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.990+0000 7fec9f61e640 1 Processor -- start 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 -- start start 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec98101c20 0x7fec981983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810d860 con 0x7fec981025e0 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810d6e0 con 0x7fec98101c20 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9f61e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec9810d9e0 con 0x7fec98100a30 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34814/0 (socket says 172.21.15.3:34814) 2024-09-12T02:20:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:50648/0 (socket says 172.21.15.3:50648) 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9dbbb640 1 -- 172.21.15.3:0/3862238711 learned_addr learned my addr 172.21.15.3:0/3862238711 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9cbb9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec98101c20 0x7fec981983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 -- 172.21.15.3:0/3862238711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec98101c20 msgr2=0x7fec981983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec98101c20 0x7fec981983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 -- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 msgr2=0x7fec9819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 -- 172.21.15.3:0/3862238711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9819fad0 con 0x7fec98100a30 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.991+0000 7fec9d3ba640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e9c0 tx=0x7fec9400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.992+0000 7fec8e7fc640 1 -- 172.21.15.3:0/3862238711 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400ccc0 con 0x7fec98100a30 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.992+0000 7fec8e7fc640 1 -- 172.21.15.3:0/3862238711 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec940035f0 con 0x7fec98100a30 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.992+0000 7fec8e7fc640 1 -- 172.21.15.3:0/3862238711 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9400ce60 con 0x7fec98100a30 2024-09-12T02:20:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.992+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9819fdc0 con 0x7fec98100a30 2024-09-12T02:20:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.992+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a0220 con 0x7fec98100a30 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.994+0000 7fec8e7fc640 1 -- 172.21.15.3:0/3862238711 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fec940037a0 con 0x7fec98100a30 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.994+0000 7feca1ffd740 1 Processor -- start 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.994+0000 7feca1ffd740 1 -- start start 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347db6f0 0x5587347e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 0x5587347e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55873473a2f0 con 0x5587347dadb0 2024-09-12T02:20:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5587346e2270 con 0x5587347db6f0 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7feca1ffd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5587347362b0 con 0x5587347e0780 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9cbb9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 0x5587347e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34820/0 (socket says 172.21.15.3:34820) 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9d3ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347db6f0 0x5587347e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 -- 172.21.15.3:0/3466586336 learned_addr learned my addr 172.21.15.3:0/3466586336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 -- 172.21.15.3:0/3466586336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 msgr2=0x5587347e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 0x5587347e2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 -- 172.21.15.3:0/3466586336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347db6f0 msgr2=0x5587347e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347db6f0 0x5587347e01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 -- 172.21.15.3:0/3466586336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587346e1340 con 0x5587347dadb0 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 0x5587347e2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:20:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec8e7fc640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 0x7fec840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9d3ba640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 0x7fec840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.995+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec900118b0 tx=0x7fec90011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3381a4d03afc648 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7fec8e7fc640 1 -- 172.21.15.3:0/3862238711 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec940cf550 con 0x7fec98100a30 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7fec837fe640 1 -- 172.21.15.3:0/3466586336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9001e070 con 0x5587347dadb0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7fec837fe640 1 -- 172.21.15.3:0/3466586336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90011310 con 0x5587347dadb0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7fec837fe640 1 -- 172.21.15.3:0/3466586336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90019e60 con 0x5587347dadb0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7feca1ffd740 1 -- 172.21.15.3:0/3466586336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 msgr2=0x5587347db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7fec9d3ba640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 0x7fec840a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec94021000 tx=0x7fec940d2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.996+0000 7feca1ffd740 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fec900118b0 tx=0x7fec90011d70 comp rx=0 tx=0).stop 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 -- 172.21.15.3:0/3466586336 shutdown_connections 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347e0780 0x5587347e2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347db6f0 0x5587347e01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 --2- 172.21.15.3:0/3466586336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db1b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 -- 172.21.15.3:0/3466586336 >> 172.21.15.3:0/3466586336 conn(0x5587347d2010 msgr2=0x5587347d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 -- 172.21.15.3:0/3466586336 shutdown_connections 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 -- 172.21.15.3:0/3466586336 wait complete. 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 -- start start 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.997+0000 7feca1ffd740 1 Processor -- start 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347ec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5587347e3750 con 0x5587347dadb0 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5587346c3290 con 0x5587347e0780 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5587347e38d0 con 0x5587347db6f0 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34828/0 (socket says 172.21.15.3:34828) 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 10 client.0 _ll_register_callbacks cb 0x5587346f4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 -- 172.21.15.3:0/2283556261 learned_addr learned my addr 172.21.15.3:0/2283556261 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347ec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 -- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 msgr2=0x5587347e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 -- 172.21.15.3:0/2283556261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 msgr2=0x5587347ec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347ec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 -- 172.21.15.3:0/2283556261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873444d210 con 0x5587347dadb0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fec90000f80 tx=0x7fec90011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6167fd625f7793 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9002d060 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec90004c50 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.998+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec90011310 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec9001e070 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 10 client.16083 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558734519620 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90011310 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fec90010cb0 con 0x5587347dadb0 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 10 client.16083 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 20 client.16083 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x5587347f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 4 client.16083 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558734747a20 con 0x5587347f0970 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:41.999+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x5587347f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.000+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x5587347f2d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3baf5fce7eb91db6 server_cookie=4791c35ec71d24d3 in_seq=0 out_seq=0 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.000+0000 7fec80ff9640 10 client.16083 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.000+0000 7fec80ff9640 1 -- 172.21.15.3:0/2283556261 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558734747a20 con 0x5587347f0970 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.000+0000 7fec80ff9640 10 client.16083 handle_command_reply: tid=0 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.000+0000 7feca1ffd740 1 client.16083 shutdown 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 msgr2=0x5587347e4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fec90000f80 tx=0x7fec90011560 comp rx=0 tx=0).stop 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 shutdown_connections 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x5587347f2d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347ec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7fec80ff9640 0 client.16083 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 --2- 172.21.15.3:0/2283556261 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347e4ef0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 >> 172.21.15.3:0/2283556261 conn(0x5587347d2010 msgr2=0x5587347d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 shutdown_connections 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 1 -- 172.21.15.3:0/2283556261 wait complete. 2024-09-12T02:20:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.001+0000 7feca1ffd740 20 client.16083 trim_cache size 0 max 16384 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.029+0000 7feca1ffd740 1 Processor -- start 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.029+0000 7feca1ffd740 1 -- start start 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347e53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a6ea0 con 0x5587347dadb0 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a8ad0 con 0x5587347e0780 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7feca1ffd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a7570 con 0x5587347db6f0 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34836/0 (socket says 172.21.15.3:34836) 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 -- 172.21.15.3:0/1738943048 learned_addr learned my addr 172.21.15.3:0/1738943048 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347e53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 -- 172.21.15.3:0/1738943048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 msgr2=0x5587347ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 -- 172.21.15.3:0/1738943048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 msgr2=0x5587347e53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347e53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 -- 172.21.15.3:0/1738943048 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587347e4a30 con 0x5587347dadb0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fec90011db0 tx=0x7fec90014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261e99492d495480 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec82ffd640 1 -- 172.21.15.3:0/1738943048 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9002d070 con 0x5587347dadb0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec82ffd640 1 -- 172.21.15.3:0/1738943048 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90011310 con 0x5587347dadb0 2024-09-12T02:20:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.030+0000 7fec82ffd640 1 -- 172.21.15.3:0/1738943048 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90037040 con 0x5587347dadb0 2024-09-12T02:20:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 -- 172.21.15.3:0/1738943048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 msgr2=0x5587347db190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fec90011db0 tx=0x7fec90014040 comp rx=0 tx=0).stop 2024-09-12T02:20:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 -- 172.21.15.3:0/1738943048 shutdown_connections 2024-09-12T02:20:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x5587347e53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347db6f0 0x5587347ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1738943048 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347dadb0 0x5587347db190 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 -- 172.21.15.3:0/1738943048 >> 172.21.15.3:0/1738943048 conn(0x5587347d2010 msgr2=0x5587347e4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 -- 172.21.15.3:0/1738943048 shutdown_connections 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.031+0000 7feca1ffd740 1 -- 172.21.15.3:0/1738943048 wait complete. 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 -- start start 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 Processor -- start 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x558734859640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a8370 con 0x5587347db6f0 2024-09-12T02:20:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a7ca0 con 0x5587347e0780 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec981a4e80 con 0x5587347dadb0 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7feca1ffd740 10 client.0 _ll_register_callbacks cb 0x7fec94005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9cbb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9cbb9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34842/0 (socket says 172.21.15.3:34842) 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9dbbb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:50690/0 (socket says 172.21.15.3:50690) 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9cbb9640 1 -- 172.21.15.3:0/1136493877 learned_addr learned my addr 172.21.15.3:0/1136493877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.032+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x558734859640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 -- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 msgr2=0x5587347e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 -- 172.21.15.3:0/1136493877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 msgr2=0x558734859640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x558734859640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 -- 172.21.15.3:0/1136493877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587347dc380 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec9cbb9640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec88009900 tx=0x7fec88000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39f1ddcb84a32db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8804a070 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec88005450 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec90011310 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec88056050 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7feca1ffd740 10 client.16089 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55873485ae10 con 0x5587347db6f0 2024-09-12T02:20:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.033+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec90011310 con 0x5587347db6f0 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fec88031a10 con 0x5587347db6f0 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 10 client.16089 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 20 client.16089 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x55873485d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 4 client.16089 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587347e0540 con 0x5587347f0970 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x55873485d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7fec80ff9640 10 client.16089 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.034+0000 7fec9dbbb640 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x55873485d870 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f229654b2ed07336 server_cookie=29e2f85ec0b4d514 in_seq=0 out_seq=0 2024-09-12T02:20:42.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 265 MiB/s wr, 27.29k op/s 2024-09-12T02:20:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:20:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:41 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 265 MiB/s wr, 27.29k op/s 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:41 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: pgmap v405: 97 pgs: 97 active+clean; 13 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 265 MiB/s wr, 27.29k op/s 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:41 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7fec80ff9640 1 -- 172.21.15.3:0/1136493877 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587347e0540 con 0x5587347f0970 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7fec80ff9640 10 client.16089 handle_command_reply: tid=0 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7feca1ffd740 1 client.16089 shutdown 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 msgr2=0x5587347e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fec88009900 tx=0x7fec88000bc0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 shutdown_connections 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.351+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5587347f0970 0x55873485d870 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7fec80ff9640 0 client.16089 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587347e0780 0x558734859640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5587347db6f0 0x5587347e99f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 --2- 172.21.15.3:0/1136493877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5587347dadb0 0x5587347e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 >> 172.21.15.3:0/1136493877 conn(0x5587347d2010 msgr2=0x5587347d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 shutdown_connections 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.352+0000 7feca1ffd740 1 -- 172.21.15.3:0/1136493877 wait complete. 2024-09-12T02:20:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.353+0000 7feca1ffd740 20 client.16089 trim_cache size 0 max 16384 2024-09-12T02:20:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:20:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:20:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:20:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:20:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.353+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 msgr2=0x7fec840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.353+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 0x7fec840a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec94021000 tx=0x7fec940d2f90 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.353+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 msgr2=0x7fec98197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.353+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e9c0 tx=0x7fec9400ee80 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 shutdown_connections 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec840a2c00 0x7fec840a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec981025e0 0x7fec9819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec98101c20 0x7fec981983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 --2- 172.21.15.3:0/3862238711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec98100a30 0x7fec98197eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 >> 172.21.15.3:0/3862238711 conn(0x7fec980fc190 msgr2=0x7fec98104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 shutdown_connections 2024-09-12T02:20:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:42.354+0000 7fec9f61e640 1 -- 172.21.15.3:0/3862238711 wait complete. 2024-09-12T02:20:43.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:20:43.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:20:43.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:42 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:43.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:20:43.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:20:43.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:42 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:43.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:20:43.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:20:43.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:42 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:20:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:44 smithi026 ceph-mon[35496]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 282 MiB/s wr, 28.39k op/s 2024-09-12T02:20:44.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:44 smithi003 ceph-mon[31476]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 282 MiB/s wr, 28.39k op/s 2024-09-12T02:20:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:44 smithi092 ceph-mon[36089]: pgmap v406: 97 pgs: 97 active+clean; 13 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 282 MiB/s wr, 28.39k op/s 2024-09-12T02:20:46.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:46 smithi026 ceph-mon[35496]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 291 MiB/s wr, 28.20k op/s 2024-09-12T02:20:46.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:46 smithi003 ceph-mon[31476]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 291 MiB/s wr, 28.20k op/s 2024-09-12T02:20:46.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:46 smithi092 ceph-mon[36089]: pgmap v407: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 291 MiB/s wr, 28.20k op/s 2024-09-12T02:20:47.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:47 smithi026 ceph-mon[35496]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 288 MiB/s wr, 27.11k op/s 2024-09-12T02:20:47.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:47 smithi003 ceph-mon[31476]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 288 MiB/s wr, 27.11k op/s 2024-09-12T02:20:47.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:47 smithi092 ceph-mon[36089]: pgmap v408: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 288 MiB/s wr, 27.11k op/s 2024-09-12T02:20:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 -- 172.21.15.3:0/4182787879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c102570 msgr2=0x7f907c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 --2- 172.21.15.3:0/4182787879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c102570 0x7f907c1029b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f906c009900 tx=0x7f906c02a4d0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 -- 172.21.15.3:0/4182787879 shutdown_connections 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 --2- 172.21.15.3:0/4182787879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102ef0 0x7f907c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 --2- 172.21.15.3:0/4182787879 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c102570 0x7f907c1029b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 --2- 172.21.15.3:0/4182787879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c100e20 0x7f907c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 -- 172.21.15.3:0/4182787879 >> 172.21.15.3:0/4182787879 conn(0x7f907c0fc7d0 msgr2=0x7f907c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 -- 172.21.15.3:0/4182787879 shutdown_connections 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.461+0000 7f9084325640 1 -- 172.21.15.3:0/4182787879 wait complete. 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 Processor -- start 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 -- start start 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c100e20 0x7f907c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c102ef0 0x7f907c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f907c10fec0 con 0x7f907c100e20 2024-09-12T02:20:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f907c10fd40 con 0x7f907c102ef0 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f9084325640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f907c110040 con 0x7f907c102570 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.462+0000 7f90820c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c100e20 0x7f907c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:50704/0 (socket says 172.21.15.3:50704) 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 -- 172.21.15.3:0/3690361941 learned_addr learned my addr 172.21.15.3:0/3690361941 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90828c2640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c102ef0 0x7f907c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 -- 172.21.15.3:0/3690361941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c102ef0 msgr2=0x7f907c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c102ef0 0x7f907c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 -- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c100e20 msgr2=0x7f907c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c100e20 0x7f907c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 -- 172.21.15.3:0/3690361941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c19f620 con 0x7f907c102570 2024-09-12T02:20:48.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.463+0000 7f90818c0640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f906c005ab0 tx=0x7f906c031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.464+0000 7f90737fe640 1 -- 172.21.15.3:0/3690361941 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c040030 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.464+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c19f8b0 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.464+0000 7f90737fe640 1 -- 172.21.15.3:0/3690361941 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c03c920 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.464+0000 7f90737fe640 1 -- 172.21.15.3:0/3690361941 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c005320 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.464+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c1a71f0 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.466+0000 7f90866fe740 1 Processor -- start 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 -- start start 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90737fe640 1 -- 172.21.15.3:0/3690361941 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f906c043030 con 0x7f907c102570 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545d4a0 0x56177545d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561775468680 0x56177546aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5617753c32e0 con 0x561775468680 2024-09-12T02:20:48.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56177536ace0 con 0x56177545ddc0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90866fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5617753bf2a0 con 0x56177545d4a0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90828c2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545d4a0 0x56177545d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90820c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90820c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:59272/0 (socket says 172.21.15.3:59272) 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90820c1640 1 -- 172.21.15.3:0/2092656803 learned_addr learned my addr 172.21.15.3:0/2092656803 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.467+0000 7f90818c0640 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561775468680 0x56177546aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 -- 172.21.15.3:0/2092656803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545d4a0 msgr2=0x56177545d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545d4a0 0x56177545d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 -- 172.21.15.3:0/2092656803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561775468680 msgr2=0x56177546aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561775468680 0x56177546aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 -- 172.21.15.3:0/2092656803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56177534f440 con 0x56177545ddc0 2024-09-12T02:20:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90737fe640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 0x7f90680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90820c1640 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f907800bd30 tx=0x7f907800b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93f1769753b1fea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f90737fe640 1 -- 172.21.15.3:0/3690361941 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f906c0ef5d0 con 0x7f907c102570 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f9070dea640 1 -- 172.21.15.3:0/2092656803 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078019020 con 0x56177545ddc0 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.468+0000 7f9070dea640 1 -- 172.21.15.3:0/2092656803 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9078015ae0 con 0x56177545ddc0 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.469+0000 7f9070dea640 1 -- 172.21.15.3:0/2092656803 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078014b30 con 0x56177545ddc0 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.469+0000 7f90866fe740 1 -- 172.21.15.3:0/2092656803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 msgr2=0x561775468140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.469+0000 7f90866fe740 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f907800bd30 tx=0x7f907800b470 comp rx=0 tx=0).stop 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90828c2640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 0x7f90680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90866fe740 1 -- 172.21.15.3:0/2092656803 shutdown_connections 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90866fe740 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x561775468680 0x56177546aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90866fe740 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x561775468140 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90866fe740 1 --2- 172.21.15.3:0/2092656803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545d4a0 0x56177545d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.470+0000 7f90866fe740 1 -- 172.21.15.3:0/2092656803 >> 172.21.15.3:0/2092656803 conn(0x561775459a30 msgr2=0x561775459e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.472+0000 7f90866fe740 1 -- 172.21.15.3:0/2092656803 shutdown_connections 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.472+0000 7f90866fe740 1 -- 172.21.15.3:0/2092656803 wait complete. 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.472+0000 7f90866fe740 1 -- start start 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.472+0000 7f90866fe740 1 Processor -- start 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x561775467f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561775468680 0x561775465730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56177546b2a0 con 0x56177545d4a0 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90818c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90818c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:59284/0 (socket says 172.21.15.3:59284) 2024-09-12T02:20:48.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90818c0640 1 -- 172.21.15.3:0/1160421452 learned_addr learned my addr 172.21.15.3:0/1160421452 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90828c2640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 0x7f90680a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9074007d90 tx=0x7f9074007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56177537e050 con 0x56177545ddc0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56177546b420 con 0x561775468680 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.473+0000 7f90828c2640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561775468680 0x561775465730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90866fe740 10 client.0 _ll_register_callbacks cb 0x5617753be160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90820c1640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x561775467f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 -- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561775468680 msgr2=0x561775465730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561775468680 0x561775465730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 -- 172.21.15.3:0/1160421452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 msgr2=0x561775467f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x561775467f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617751343c0 con 0x56177545ddc0 2024-09-12T02:20:48.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.474+0000 7f90818c0640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f906c0bbbd0 tx=0x7f906c02e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd3ae73a575c95b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.475+0000 7f9065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.475+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c04a070 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.475+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9078015ae0 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.475+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f906c0ba1d0 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.475+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f906c0c01d0 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.477+0000 7f90866fe740 10 client.25583 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.477+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56177547f400 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.478+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c0c4a90 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.478+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f906c0054c0 con 0x56177545ddc0 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 10 client.25583 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:48.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 20 client.25583 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x561775480a80 0x561775482ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 4 client.25583 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9078015ae0 con 0x561775480a80 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.479+0000 7f90820c1640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x561775480a80 0x561775482ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.480+0000 7f90820c1640 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x561775480a80 0x561775482ec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e5fb613c1e973b server_cookie=4366bddf8edd8cd7 in_seq=0 out_seq=0 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.480+0000 7f9065ffb640 10 client.25583 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.481+0000 7f9065ffb640 1 -- 172.21.15.3:0/1160421452 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9078015ae0 con 0x561775480a80 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f9065ffb640 10 client.25583 handle_command_reply: tid=0 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 client.25583 shutdown 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 msgr2=0x5617754651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f906c0bbbd0 tx=0x7f906c02e000 comp rx=0 tx=0).stop 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 shutdown_connections 2024-09-12T02:20:48.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x561775480a80 0x561775482ec0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561775468680 0x561775465730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56177545ddc0 0x5617754651f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 --2- 172.21.15.3:0/1160421452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x561775467f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.482+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 >> 172.21.15.3:0/1160421452 conn(0x561775459a30 msgr2=0x561775458b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.483+0000 7f9065ffb640 0 client.25583 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.483+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 shutdown_connections 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.483+0000 7f90866fe740 1 -- 172.21.15.3:0/1160421452 wait complete. 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.483+0000 7f90866fe740 20 client.25583 trim_cache size 0 max 16384 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.506+0000 7f90866fe740 1 Processor -- start 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.506+0000 7f90866fe740 1 -- start start 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.506+0000 7f90866fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.506+0000 7f90866fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x561775488d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90866fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x561775458350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90866fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ab520 con 0x56177545d4a0 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90866fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ad150 con 0x561775468680 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90866fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1abbf0 con 0x56177545ddc0 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34876/0 (socket says 172.21.15.3:34876) 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 -- 172.21.15.3:0/1212115080 learned_addr learned my addr 172.21.15.3:0/1212115080 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90818c0640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x561775488d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90828c2640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x561775458350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 -- 172.21.15.3:0/1212115080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 msgr2=0x561775488d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x561775488d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 -- 172.21.15.3:0/1212115080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 msgr2=0x561775458350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x561775458350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 -- 172.21.15.3:0/1212115080 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561775463740 con 0x56177545d4a0 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90828c2640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x561775458350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f90820c1640 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f907800d010 tx=0x7f9078014360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7e89b969112cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.507+0000 7f9067fff640 1 -- 172.21.15.3:0/1212115080 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078019040 con 0x56177545d4a0 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.508+0000 7f9067fff640 1 -- 172.21.15.3:0/1212115080 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9078014480 con 0x56177545d4a0 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.508+0000 7f9067fff640 1 -- 172.21.15.3:0/1212115080 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078014a40 con 0x56177545d4a0 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- 172.21.15.3:0/1212115080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 msgr2=0x5617754887d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f907800d010 tx=0x7f9078014360 comp rx=0 tx=0).stop 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- 172.21.15.3:0/1212115080 shutdown_connections 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x561775458350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x561775488d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 --2- 172.21.15.3:0/1212115080 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56177545d4a0 0x5617754887d0 secure :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f907800d010 tx=0x7f9078014360 comp rx=0 tx=0).stop 2024-09-12T02:20:48.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- 172.21.15.3:0/1212115080 >> 172.21.15.3:0/1212115080 conn(0x561775459a30 msgr2=0x5617754608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- 172.21.15.3:0/1212115080 shutdown_connections 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- 172.21.15.3:0/1212115080 wait complete. 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.509+0000 7f90866fe740 1 -- start start 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.510+0000 7f90866fe740 1 Processor -- start 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x5617754792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x5617754797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90820c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x5617754792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5617754746e0 0x561775474b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90820c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x5617754792a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:50736/0 (socket says 172.21.15.3:50736) 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ac9f0 con 0x5617754746e0 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1ac320 con 0x561775468680 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1a9540 con 0x56177545ddc0 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90820c1640 1 -- 172.21.15.3:0/1584819935 learned_addr learned my addr 172.21.15.3:0/1584819935 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:48.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90818c0640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x5617754797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90828c2640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5617754746e0 0x561775474b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.511+0000 7f90866fe740 10 client.0 _ll_register_callbacks cb 0x7f906c0f2f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 -- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 msgr2=0x5617754792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x5617754792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 -- 172.21.15.3:0/1584819935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5617754746e0 msgr2=0x561775474b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5617754746e0 0x561775474b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 -- 172.21.15.3:0/1584819935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561775475390 con 0x561775468680 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90818c0640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x5617754797e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5617754667a0 tx=0x7f906c0c4600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06bbae0ffc7d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f9065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c04a040 con 0x561775468680 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f906c0c46d0 con 0x561775468680 2024-09-12T02:20:48.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.512+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9078014480 con 0x561775468680 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.513+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f906c0b8540 con 0x561775468680 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.513+0000 7f90866fe740 10 client.25589 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.513+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5617754eb780 con 0x561775468680 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.513+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9078014480 con 0x561775468680 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f906c0bb320 con 0x561775468680 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 10 client.25589 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:48.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 20 client.25589 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5617754ebd70 0x5617754ee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 4 client.25589 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.514+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617753d0850 con 0x5617754ebd70 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.516+0000 7f90820c1640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5617754ebd70 0x5617754ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.518+0000 7f90820c1640 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5617754ebd70 0x5617754ee190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74763756287ec090 server_cookie=68cabc73890b479f in_seq=0 out_seq=0 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.518+0000 7f9065ffb640 10 client.25589 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f9065ffb640 1 -- 172.21.15.3:0/1584819935 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5617753d0850 con 0x5617754ebd70 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:20:48.844 INFO:teuthology.orchestra.run.smithi003.stdout: "status": "no active scrubs running", 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stdout: "scrubs": {} 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f9065ffb640 10 client.25589 handle_command_reply: tid=0 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f90866fe740 1 client.25589 shutdown 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 msgr2=0x5617754797e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x5617754797e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5617754667a0 tx=0x7f906c0c4600 comp rx=0 tx=0).stop 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 shutdown_connections 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.519+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5617754ebd70 0x5617754ee190 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f9065ffb640 0 client.25589 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5617754746e0 0x561775474b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561775468680 0x5617754797e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 --2- 172.21.15.3:0/1584819935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56177545ddc0 0x5617754792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 >> 172.21.15.3:0/1584819935 conn(0x561775459a30 msgr2=0x561775458350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 shutdown_connections 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.520+0000 7f90866fe740 1 -- 172.21.15.3:0/1584819935 wait complete. 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.521+0000 7f90866fe740 20 client.25589 trim_cache size 0 max 16384 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 msgr2=0x7f90680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 0x7f90680a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9074007d90 tx=0x7f9074007520 comp rx=0 tx=0).stop 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 msgr2=0x7f907c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f906c005ab0 tx=0x7f906c031ba0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 shutdown_connections 2024-09-12T02:20:48.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f90680a2c00 0x7f90680a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f907c102ef0 0x7f907c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f907c102570 0x7f907c19abb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 --2- 172.21.15.3:0/3690361941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f907c100e20 0x7f907c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.526+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 >> 172.21.15.3:0/3690361941 conn(0x7f907c0fc7d0 msgr2=0x7f907c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.527+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 shutdown_connections 2024-09-12T02:20:48.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:20:47.527+0000 7f9084325640 1 -- 172.21.15.3:0/3690361941 wait complete. 2024-09-12T02:20:50.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:49 smithi026 ceph-mon[35496]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 275 MiB/s wr, 25.74k op/s 2024-09-12T02:20:50.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:49 smithi003 ceph-mon[31476]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 275 MiB/s wr, 25.74k op/s 2024-09-12T02:20:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:49 smithi092 ceph-mon[36089]: pgmap v409: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 275 MiB/s wr, 25.74k op/s 2024-09-12T02:20:52.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:51 smithi026 ceph-mon[35496]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 273 MiB/s wr, 25.81k op/s 2024-09-12T02:20:52.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:51 smithi003 ceph-mon[31476]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 273 MiB/s wr, 25.81k op/s 2024-09-12T02:20:52.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:51 smithi092 ceph-mon[36089]: pgmap v410: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 273 MiB/s wr, 25.81k op/s 2024-09-12T02:20:53.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:53 smithi003 ceph-mon[31476]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 243 MiB/s wr, 23.51k op/s 2024-09-12T02:20:54.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:53 smithi026 ceph-mon[35496]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 243 MiB/s wr, 23.51k op/s 2024-09-12T02:20:54.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:53 smithi092 ceph-mon[36089]: pgmap v411: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 243 MiB/s wr, 23.51k op/s 2024-09-12T02:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.519+0000 7fec44981640 1 -- 172.21.15.3:0/2032006971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 msgr2=0x7fec401020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.519+0000 7fec44981640 1 --2- 172.21.15.3:0/2032006971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec401020b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec3401a910 tx=0x7fec3403b470 comp rx=0 tx=0).stop 2024-09-12T02:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.521+0000 7fec44981640 1 -- 172.21.15.3:0/2032006971 shutdown_connections 2024-09-12T02:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.521+0000 7fec44981640 1 --2- 172.21.15.3:0/2032006971 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec401025f0 0x7fec40108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.521+0000 7fec44981640 1 --2- 172.21.15.3:0/2032006971 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec401020b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.521+0000 7fec44981640 1 --2- 172.21.15.3:0/2032006971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec40100a40 0x7fec40100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.521+0000 7fec44981640 1 -- 172.21.15.3:0/2032006971 >> 172.21.15.3:0/2032006971 conn(0x7fec400fc1c0 msgr2=0x7fec400fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.524+0000 7fec44981640 1 -- 172.21.15.3:0/2032006971 shutdown_connections 2024-09-12T02:20:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.527+0000 7fec44981640 1 -- 172.21.15.3:0/2032006971 wait complete. 2024-09-12T02:20:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.530+0000 7fec44981640 1 Processor -- start 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 -- start start 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec40073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401025f0 0x7fec40071db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec4006c640 con 0x7fec40100a40 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec4006c4c0 con 0x7fec401025f0 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec44981640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec4006c7c0 con 0x7fec40101c30 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec3e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec3e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59354/0 (socket says 172.21.15.3:59354) 2024-09-12T02:20:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec3e59c640 1 -- 172.21.15.3:0/2629723951 learned_addr learned my addr 172.21.15.3:0/2629723951 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.531+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec40073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401025f0 0x7fec40071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 -- 172.21.15.3:0/2629723951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 msgr2=0x7fec40073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec40073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 -- 172.21.15.3:0/2629723951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401025f0 msgr2=0x7fec40071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401025f0 0x7fec40071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 -- 172.21.15.3:0/2629723951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec40072580 con 0x7fec40100a40 2024-09-12T02:20:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.532+0000 7fec3e59c640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec3000d770 tx=0x7fec3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.533+0000 7fec2f7fe640 1 -- 172.21.15.3:0/2629723951 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30017070 con 0x7fec40100a40 2024-09-12T02:20:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.533+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec40072870 con 0x7fec40100a40 2024-09-12T02:20:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.533+0000 7fec2f7fe640 1 -- 172.21.15.3:0/2629723951 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30009c30 con 0x7fec40100a40 2024-09-12T02:20:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.533+0000 7fec2f7fe640 1 -- 172.21.15.3:0/2629723951 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30017070 con 0x7fec40100a40 2024-09-12T02:20:55.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.537+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401a6b60 con 0x7fec40100a40 2024-09-12T02:20:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.547+0000 7fec477f4740 1 Processor -- start 2024-09-12T02:20:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.548+0000 7fec2f7fe640 1 -- 172.21.15.3:0/2629723951 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fec30005070 con 0x7fec40100a40 2024-09-12T02:20:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.548+0000 7fec477f4740 1 -- start start 2024-09-12T02:20:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.549+0000 7fec2f7fe640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 0x7fec280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.549+0000 7fec2f7fe640 1 -- 172.21.15.3:0/2629723951 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec30009a30 con 0x7fec40100a40 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564eab31e2f0 con 0x564eab3c0370 2024-09-12T02:20:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564eab2c6270 con 0x564eab3ba9a0 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.550+0000 7fec477f4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564eab31a2b0 con 0x564eab3bb2e0 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.551+0000 7fec3dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.551+0000 7fec3dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:43942/0 (socket says 172.21.15.3:43942) 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.551+0000 7fec3dd9b640 1 -- 172.21.15.3:0/3147734245 learned_addr learned my addr 172.21.15.3:0/3147734245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.552+0000 7fec3e59c640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.552+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.554+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 0x7fec280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3ed9d640 1 -- 172.21.15.3:0/3147734245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 msgr2=0x564eab3bfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3bfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3ed9d640 1 -- 172.21.15.3:0/3147734245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 msgr2=0x564eab3bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3ed9d640 1 -- 172.21.15.3:0/3147734245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eab2c5340 con 0x564eab3c0370 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.556+0000 7fec3e59c640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:20:55.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.557+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c2830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec3800ae60 tx=0x7fec38016640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7b5eba4726557f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.557+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 0x7fec280a5500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec3800d000 tx=0x7fec3801a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.561+0000 7fec2cff9640 1 -- 172.21.15.3:0/3147734245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38026070 con 0x564eab3c0370 2024-09-12T02:20:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.561+0000 7fec2cff9640 1 -- 172.21.15.3:0/3147734245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3801d030 con 0x564eab3c0370 2024-09-12T02:20:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.561+0000 7fec2cff9640 1 -- 172.21.15.3:0/3147734245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38013040 con 0x564eab3c0370 2024-09-12T02:20:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.564+0000 7fec477f4740 1 -- 172.21.15.3:0/3147734245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 msgr2=0x564eab3c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.564+0000 7fec477f4740 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c2830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec3800ae60 tx=0x7fec38016640 comp rx=0 tx=0).stop 2024-09-12T02:20:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.565+0000 7fec477f4740 1 -- 172.21.15.3:0/3147734245 shutdown_connections 2024-09-12T02:20:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.565+0000 7fec477f4740 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c2830 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.565+0000 7fec477f4740 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.565+0000 7fec477f4740 1 --2- 172.21.15.3:0/3147734245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.565+0000 7fec477f4740 1 -- 172.21.15.3:0/3147734245 >> 172.21.15.3:0/3147734245 conn(0x564eab3b6010 msgr2=0x564eab3b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.571+0000 7fec477f4740 1 -- 172.21.15.3:0/3147734245 shutdown_connections 2024-09-12T02:20:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.573+0000 7fec477f4740 1 -- 172.21.15.3:0/3147734245 wait complete. 2024-09-12T02:20:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.573+0000 7fec477f4740 1 -- start start 2024-09-12T02:20:55.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.574+0000 7fec477f4740 1 Processor -- start 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564eab3c32e0 con 0x564eab3c0370 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564eab2a7290 con 0x564eab3ba9a0 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564eab3c3460 con 0x564eab3bb2e0 2024-09-12T02:20:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.577+0000 7fec477f4740 10 client.0 _ll_register_callbacks cb 0x564eab2d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.578+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.578+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59382/0 (socket says 172.21.15.3:59382) 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.578+0000 7fec3ed9d640 1 -- 172.21.15.3:0/2861998882 learned_addr learned my addr 172.21.15.3:0/2861998882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.579+0000 7fec3ed9d640 1 -- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 msgr2=0x564eab3c5130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.579+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3c5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.579+0000 7fec3ed9d640 1 -- 172.21.15.3:0/2861998882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 msgr2=0x564eab3c4bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.579+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3c4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.579+0000 7fec3ed9d640 1 -- 172.21.15.3:0/2861998882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eab031210 con 0x564eab3c0370 2024-09-12T02:20:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.581+0000 7fec3ed9d640 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec38009c50 tx=0x7fec38019c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358dcd068ae8519c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.582+0000 7fec25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.587+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38026070 con 0x564eab3c0370 2024-09-12T02:20:55.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.587+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec38016e10 con 0x564eab3c0370 2024-09-12T02:20:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.588+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3801d030 con 0x564eab3c0370 2024-09-12T02:20:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.593+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec3800aa00 con 0x564eab3c0370 2024-09-12T02:20:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.596+0000 7fec477f4740 10 client.16116 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.596+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564eab0fd620 con 0x564eab3c0370 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3801d030 con 0x564eab3c0370 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fec38021420 con 0x564eab3c0370 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 10 client.16116 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 20 client.16116 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab3d0c60 0x564eab3d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 4 client.16116 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:20:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eab32ba20 con 0x564eab3d0c60 2024-09-12T02:20:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.598+0000 7fec3e59c640 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab3d0c60 0x564eab3d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.600+0000 7fec3e59c640 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab3d0c60 0x564eab3d30e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126a5c85401763a2 server_cookie=520174b7f3273fe7 in_seq=0 out_seq=0 2024-09-12T02:20:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.601+0000 7fec25ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.603+0000 7fec25ffb640 1 -- 172.21.15.3:0/2861998882 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564eab32ba20 con 0x564eab3d0c60 2024-09-12T02:20:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.603+0000 7fec25ffb640 10 client.16116 handle_command_reply: tid=0 2024-09-12T02:20:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.603+0000 7fec477f4740 1 client.16116 shutdown 2024-09-12T02:20:55.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 msgr2=0x564eab3cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fec38009c50 tx=0x7fec38019c50 comp rx=0 tx=0).stop 2024-09-12T02:20:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 shutdown_connections 2024-09-12T02:20:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab3d0c60 0x564eab3d30e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3cc170 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 --2- 172.21.15.3:0/2861998882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ba9a0 0x564eab3c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 >> 172.21.15.3:0/2861998882 conn(0x564eab3b6010 msgr2=0x564eab3b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.606+0000 7fec25ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.607+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 shutdown_connections 2024-09-12T02:20:55.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.607+0000 7fec477f4740 1 -- 172.21.15.3:0/2861998882 wait complete. 2024-09-12T02:20:55.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.611+0000 7fec477f4740 20 client.16116 trim_cache size 0 max 16384 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.653+0000 7fec477f4740 1 Processor -- start 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 -- start start 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3ba9a0 0x564eab3b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3c0370 0x564eab3b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec401af950 con 0x564eab3ba9a0 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b1580 con 0x564eab3c0370 2024-09-12T02:20:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec477f4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b0020 con 0x564eab3bb2e0 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:43962/0 (socket says 172.21.15.3:43962) 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 -- 172.21.15.3:0/666991905 learned_addr learned my addr 172.21.15.3:0/666991905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 -- 172.21.15.3:0/666991905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3c0370 msgr2=0x564eab3b7400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3c0370 0x564eab3b7400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 -- 172.21.15.3:0/666991905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3ba9a0 msgr2=0x564eab3b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3ba9a0 0x564eab3b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.654+0000 7fec3dd9b640 1 -- 172.21.15.3:0/666991905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eab3b5e30 con 0x564eab3bb2e0 2024-09-12T02:20:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.655+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec3401a860 tx=0x7fec34004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35204de8e36ad11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.655+0000 7fec27fff640 1 -- 172.21.15.3:0/666991905 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34046030 con 0x564eab3bb2e0 2024-09-12T02:20:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.655+0000 7fec27fff640 1 -- 172.21.15.3:0/666991905 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec34052070 con 0x564eab3bb2e0 2024-09-12T02:20:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.655+0000 7fec27fff640 1 -- 172.21.15.3:0/666991905 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34040870 con 0x564eab3bb2e0 2024-09-12T02:20:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.657+0000 7fec477f4740 1 -- 172.21.15.3:0/666991905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 msgr2=0x564eab3cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.657+0000 7fec477f4740 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec3401a860 tx=0x7fec34004770 comp rx=0 tx=0).stop 2024-09-12T02:20:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 -- 172.21.15.3:0/666991905 shutdown_connections 2024-09-12T02:20:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3c0370 0x564eab3b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3bb2e0 0x564eab3cc130 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec3401a860 tx=0x7fec34004770 comp rx=0 tx=0).stop 2024-09-12T02:20:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 --2- 172.21.15.3:0/666991905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3ba9a0 0x564eab3b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 -- 172.21.15.3:0/666991905 >> 172.21.15.3:0/666991905 conn(0x564eab3b6010 msgr2=0x564eab3b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.659+0000 7fec477f4740 1 -- 172.21.15.3:0/666991905 shutdown_connections 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.660+0000 7fec477f4740 1 -- 172.21.15.3:0/666991905 wait complete. 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.660+0000 7fec477f4740 1 -- start start 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 Processor -- start 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3ba9a0 0x564eab3c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 0x564eab3cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b0e20 con 0x564eab3c0370 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b0750 con 0x564eab3ca860 2024-09-12T02:20:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fec401ad930 con 0x564eab3ba9a0 2024-09-12T02:20:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.661+0000 7fec477f4740 10 client.0 _ll_register_callbacks cb 0x7fec38024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:20:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.662+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 0x564eab3cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.663+0000 7fec3ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 0x564eab3cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:36276/0 (socket says 172.21.15.3:36276) 2024-09-12T02:20:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.663+0000 7fec3ed9d640 1 -- 172.21.15.3:0/1552524963 learned_addr learned my addr 172.21.15.3:0/1552524963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:20:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.663+0000 7fec3e59c640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3ba9a0 0x564eab3c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.663+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.664+0000 7fec3dd9b640 1 -- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3ba9a0 msgr2=0x564eab3c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.664+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3ba9a0 0x564eab3c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.664+0000 7fec3dd9b640 1 -- 172.21.15.3:0/1552524963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 msgr2=0x564eab3cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:20:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.664+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 0x564eab3cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.664+0000 7fec3dd9b640 1 -- 172.21.15.3:0/1552524963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eab3cb250 con 0x564eab3c0370 2024-09-12T02:20:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.665+0000 7fec3dd9b640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c9850 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x564eab3c2ca0 tx=0x7fec3401a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298395bdd5a9ef28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:20:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.665+0000 7fec25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:20:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.665+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34004250 con 0x564eab3c0370 2024-09-12T02:20:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.665+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fec340043f0 con 0x564eab3c0370 2024-09-12T02:20:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.665+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec34052070 con 0x564eab3c0370 2024-09-12T02:20:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.666+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec34004f30 con 0x564eab3c0370 2024-09-12T02:20:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.666+0000 7fec477f4740 10 client.16119 fetch_fsmap learned FSMap version 33 2024-09-12T02:20:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.666+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564eab43b9c0 con 0x564eab3c0370 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec34052070 con 0x564eab3c0370 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fec3404b040 con 0x564eab3c0370 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 10 client.16119 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 20 client.16119 populate_metadata read hostname 'smithi003' 2024-09-12T02:20:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab43c170 0x564eab43e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:20:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 4 client.16119 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:20:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564eab325d70 con 0x564eab43c170 2024-09-12T02:20:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.667+0000 7fec3e59c640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab43c170 0x564eab43e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:20:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.668+0000 7fec25ffb640 10 client.16119 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:55.668+0000 7fec3e59c640 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab43c170 0x564eab43e590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=686fa5e15229c968 server_cookie=2c95ae067b525269 in_seq=0 out_seq=0 2024-09-12T02:20:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:55 smithi026 ceph-mon[35496]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 240 MiB/s wr, 24.83k op/s 2024-09-12T02:20:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:55 smithi003 ceph-mon[31476]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 240 MiB/s wr, 24.83k op/s 2024-09-12T02:20:56.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:55 smithi092 ceph-mon[36089]: pgmap v412: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 240 MiB/s wr, 24.83k op/s 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.399+0000 7fec25ffb640 1 -- 172.21.15.3:0/1552524963 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564eab325d70 con 0x564eab43c170 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.399+0000 7fec25ffb640 10 client.16119 handle_command_reply: tid=0 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.399+0000 7fec477f4740 1 client.16119 shutdown 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.399+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 msgr2=0x564eab3c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.399+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c9850 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x564eab3c2ca0 tx=0x7fec3401a270 comp rx=0 tx=0).stop 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 shutdown_connections 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564eab43c170 0x564eab43e590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564eab3ca860 0x564eab3cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564eab3c0370 0x564eab3c9850 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 --2- 172.21.15.3:0/1552524963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564eab3ba9a0 0x564eab3c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec25ffb640 0 client.16119 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 >> 172.21.15.3:0/1552524963 conn(0x564eab3b6010 msgr2=0x564eab3b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 shutdown_connections 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 1 -- 172.21.15.3:0/1552524963 wait complete. 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.400+0000 7fec477f4740 20 client.16119 trim_cache size 0 max 16384 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:20:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:20:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:20:56.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.402+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 msgr2=0x7fec280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.402+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 0x7fec280a5500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fec3800d000 tx=0x7fec3801a000 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.402+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 msgr2=0x7fec40073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.402+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec3000d770 tx=0x7fec3000dc30 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 shutdown_connections 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fec280a3040 0x7fec280a5500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec401025f0 0x7fec40071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec40101c30 0x7fec40073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 --2- 172.21.15.3:0/2629723951 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec40100a40 0x7fec40073770 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:20:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 >> 172.21.15.3:0/2629723951 conn(0x7fec400fc1c0 msgr2=0x7fec40104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:20:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 shutdown_connections 2024-09-12T02:20:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:20:56.403+0000 7fec44981640 1 -- 172.21.15.3:0/2629723951 wait complete. 2024-09-12T02:20:58.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:57 smithi026 ceph-mon[35496]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 207 MiB/s wr, 22.86k op/s 2024-09-12T02:20:58.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:57 smithi003 ceph-mon[31476]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 207 MiB/s wr, 22.86k op/s 2024-09-12T02:20:58.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:57 smithi092 ceph-mon[36089]: pgmap v413: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 207 MiB/s wr, 22.86k op/s 2024-09-12T02:21:00.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:20:59 smithi026 ceph-mon[35496]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 181 MiB/s wr, 21.26k op/s 2024-09-12T02:21:00.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:20:59 smithi003 ceph-mon[31476]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 181 MiB/s wr, 21.26k op/s 2024-09-12T02:21:00.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:20:59 smithi092 ceph-mon[36089]: pgmap v414: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 181 MiB/s wr, 21.26k op/s 2024-09-12T02:21:00.407 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:21:00.407 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:21:00.444 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:02.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:01 smithi026 ceph-mon[35496]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 176 MiB/s wr, 21.68k op/s 2024-09-12T02:21:02.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:01 smithi003 ceph-mon[31476]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 176 MiB/s wr, 21.68k op/s 2024-09-12T02:21:02.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:01 smithi092 ceph-mon[36089]: pgmap v415: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 176 MiB/s wr, 21.68k op/s 2024-09-12T02:21:04.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:03 smithi026 ceph-mon[35496]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 150 MiB/s wr, 19.20k op/s 2024-09-12T02:21:04.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:03 smithi003 ceph-mon[31476]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 150 MiB/s wr, 19.20k op/s 2024-09-12T02:21:04.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:03 smithi092 ceph-mon[36089]: pgmap v416: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 150 MiB/s wr, 19.20k op/s 2024-09-12T02:21:05.871 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:05 smithi003 ceph-mon[31476]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 154 MiB/s wr, 20.15k op/s 2024-09-12T02:21:06.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:05 smithi026 ceph-mon[35496]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 154 MiB/s wr, 20.15k op/s 2024-09-12T02:21:06.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:05 smithi092 ceph-mon[36089]: pgmap v417: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 154 MiB/s wr, 20.15k op/s 2024-09-12T02:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 -- 172.21.15.3:0/849277881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc1036e0 msgr2=0x7fdbbc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 --2- 172.21.15.3:0/849277881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc1036e0 0x7fdbbc107b90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009890 tx=0x7fdbac02a3b0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 -- 172.21.15.3:0/849277881 shutdown_connections 2024-09-12T02:21:07.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 --2- 172.21.15.3:0/849277881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 --2- 172.21.15.3:0/849277881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc1036e0 0x7fdbbc107b90 secure :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009890 tx=0x7fdbac02a3b0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 --2- 172.21.15.3:0/849277881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc102d30 0x7fdbbc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 -- 172.21.15.3:0/849277881 >> 172.21.15.3:0/849277881 conn(0x7fdbbc0fc7d0 msgr2=0x7fdbbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.766+0000 7fdbc395e640 1 -- 172.21.15.3:0/849277881 shutdown_connections 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.768+0000 7fdbc395e640 1 -- 172.21.15.3:0/849277881 wait complete. 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.769+0000 7fdbc395e640 1 Processor -- start 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.769+0000 7fdbc395e640 1 -- start start 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc395e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc102d30 0x7fdbbc19e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc395e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc395e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc198660 0x7fdbbc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc395e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112090 con 0x7fdbbc102d30 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc395e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc111f10 con 0x7fdbbc198660 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc0ef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc0ef9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:51020/0 (socket says 172.21.15.3:51020) 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc0ef9640 1 -- 172.21.15.3:0/2899246520 learned_addr learned my addr 172.21.15.3:0/2899246520 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc102d30 0x7fdbbc19e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.770+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc198660 0x7fdbbc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.771+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc112210 con 0x7fdbbc1082a0 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.773+0000 7fdbc0ef9640 1 -- 172.21.15.3:0/2899246520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc198660 msgr2=0x7fdbbc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.773+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc198660 0x7fdbbc198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.773+0000 7fdbc0ef9640 1 -- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc102d30 msgr2=0x7fdbbc19e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.773+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc102d30 0x7fdbbc19e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.773+0000 7fdbc0ef9640 1 -- 172.21.15.3:0/2899246520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc199210 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.774+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbac035040 tx=0x7fdbac033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.776+0000 7fdbb27fc640 1 -- 172.21.15.3:0/2899246520 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac037040 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.776+0000 7fdbb27fc640 1 -- 172.21.15.3:0/2899246520 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbac0305c0 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.776+0000 7fdbb27fc640 1 -- 172.21.15.3:0/2899246520 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0055c0 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.776+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1994a0 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.783+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a5570 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.789+0000 7fdbc5d37740 1 Processor -- start 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.791+0000 7fdbb27fc640 1 -- 172.21.15.3:0/2899246520 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdbac042060 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.793+0000 7fdbb27fc640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 0x7fdba80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.793+0000 7fdbc5d37740 1 -- start start 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.794+0000 7fdbb27fc640 1 -- 172.21.15.3:0/2899246520 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdbac0ede70 con 0x7fdbbc1082a0 2024-09-12T02:21:07.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.794+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.794+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 0x7fdba80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.795+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.795+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 0x559391e9d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.795+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559391df12a0 con 0x559391e8fb70 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc0ef9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc0ef9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52588/0 (socket says 172.21.15.3:52588) 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc0ef9640 1 -- 172.21.15.3:0/2378452375 learned_addr learned my addr 172.21.15.3:0/2378452375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559391d98b10 con 0x559391e90490 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559391ded260 con 0x559391e9ad50 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.796+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 0x559391e9d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.798+0000 7fdbc1efb640 1 -- 172.21.15.3:0/2378452375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 msgr2=0x559391e9a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.798+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.798+0000 7fdbc1efb640 1 -- 172.21.15.3:0/2378452375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 msgr2=0x559391e8ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.798+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.799+0000 7fdbc1efb640 1 -- 172.21.15.3:0/2378452375 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559391b62b90 con 0x559391e9ad50 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.799+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.799+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 0x7fdba80a53f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005fd0 tx=0x7fdbb4014b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.799+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 0x559391e9d140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4010740 tx=0x7fdbb401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=895e3c506e1cf938 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.800+0000 7fdba77fe640 1 -- 172.21.15.3:0/2378452375 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4003970 con 0x559391e9ad50 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.803+0000 7fdba77fe640 1 -- 172.21.15.3:0/2378452375 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb401e730 con 0x559391e9ad50 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.803+0000 7fdba77fe640 1 -- 172.21.15.3:0/2378452375 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4004350 con 0x559391e9ad50 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 msgr2=0x559391e9d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 0x559391e9d140 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4010740 tx=0x7fdbb401e000 comp rx=0 tx=0).stop 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 shutdown_connections 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e9ad50 0x559391e9d140 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/2378452375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e8ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.804+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 >> 172.21.15.3:0/2378452375 conn(0x559391e8c100 msgr2=0x559391e8c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.805+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 shutdown_connections 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.805+0000 7fdbc5d37740 1 -- 172.21.15.3:0/2378452375 wait complete. 2024-09-12T02:21:07.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.805+0000 7fdbc5d37740 1 -- start start 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.805+0000 7fdbc5d37740 1 Processor -- start 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.806+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.806+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391e98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e9ad50 0x559391e98560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc16fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc16fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:51044/0 (socket says 172.21.15.3:51044) 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc16fa640 1 -- 172.21.15.3:0/3492251549 learned_addr learned my addr 172.21.15.3:0/3492251549 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x559391e9d960 con 0x559391e90490 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e9ad50 0x559391e98560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x559391d796e0 con 0x559391e9ad50 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559391e9dae0 con 0x559391e8fb70 2024-09-12T02:21:07.098 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.807+0000 7fdbc5d37740 10 client.0 _ll_register_callbacks cb 0x559391d9da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 -- 172.21.15.3:0/3492251549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e9ad50 msgr2=0x559391e98560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e9ad50 0x559391e98560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 -- 172.21.15.3:0/3492251549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 msgr2=0x559391e98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391e98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559391ea6e20 con 0x559391e8fb70 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800b8f0 tx=0x7fdbb800bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7ebddf55ce9fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.808+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb401e730 con 0x559391e8fb70 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdba4ff9640 10 client.25732 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdba4ff9640 1 -- 172.21.15.3:0/3492251549 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb801a470 con 0x559391e8fb70 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdba4ff9640 1 -- 172.21.15.3:0/3492251549 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdbb801a610 con 0x559391e8fb70 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdba4ff9640 1 -- 172.21.15.3:0/3492251549 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdbb801a850 con 0x559391e8fb70 2024-09-12T02:21:07.099 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdbc5d37740 10 client.25732 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.810+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({fsmap=33}) -- 0x559391ea9590 con 0x559391e8fb70 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.812+0000 7fdba4ff9640 1 -- 172.21.15.3:0/3492251549 <== mon.1 v2:172.21.15.92:3300/0 4 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdbb8011440 con 0x559391e8fb70 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 10 client.25732 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 10 client.25732 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 20 client.25732 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391eaa940 0x559391eacd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 4 client.25732 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.813+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdbb401e730 con 0x559391eaa940 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.815+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391eaa940 0x559391eacd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.818+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391eaa940 0x559391eacd50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a543638944cfdf server_cookie=b4da4b0f7b8567ae in_seq=0 out_seq=0 2024-09-12T02:21:07.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.818+0000 7fdba4ff9640 10 client.25732 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.819+0000 7fdba4ff9640 1 -- 172.21.15.3:0/3492251549 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fdbb401e730 con 0x559391eaa940 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.819+0000 7fdba4ff9640 10 client.25732 handle_command_reply: tid=0 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.819+0000 7fdbc5d37740 1 client.25732 shutdown 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 msgr2=0x559391e97ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800b8f0 tx=0x7fdbb800bdb0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 shutdown_connections 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391eaa940 0x559391eacd50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e9ad50 0x559391e98560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391e98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/3492251549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e97ae0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 >> 172.21.15.3:0/3492251549 conn(0x559391e8c100 msgr2=0x559391e86e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:07.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.822+0000 7fdba4ff9640 0 client.25732 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.825+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 shutdown_connections 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.825+0000 7fdbc5d37740 1 -- 172.21.15.3:0/3492251549 wait complete. 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.826+0000 7fdbc5d37740 20 client.25732 trim_cache size 0 max 16384 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.869+0000 7fdbc5d37740 1 Processor -- start 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.869+0000 7fdbc5d37740 1 -- start start 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e8c100 0x559391e86cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1aba80 con 0x559391e90490 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1ad6b0 con 0x559391e8c100 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.870+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1ac150 con 0x559391e8fb70 2024-09-12T02:21:07.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.871+0000 7fdbc16fa640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e8c100 0x559391e86cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.871+0000 7fdbc1efb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.872+0000 7fdbc1efb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36704/0 (socket says 172.21.15.3:36704) 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.872+0000 7fdbc1efb640 1 -- 172.21.15.3:0/670435053 learned_addr learned my addr 172.21.15.3:0/670435053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.873+0000 7fdbc1efb640 1 -- 172.21.15.3:0/670435053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 msgr2=0x559391e963b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.873+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e963b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.873+0000 7fdbc1efb640 1 -- 172.21.15.3:0/670435053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e8c100 msgr2=0x559391e86cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.873+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e8c100 0x559391e86cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.873+0000 7fdbc1efb640 1 -- 172.21.15.3:0/670435053 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559391e8bf30 con 0x559391e90490 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.877+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdbb401efd0 tx=0x7fdbb4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c9821d91d824e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.878+0000 7fdba6ffd640 1 -- 172.21.15.3:0/670435053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb402e030 con 0x559391e90490 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.878+0000 7fdba6ffd640 1 -- 172.21.15.3:0/670435053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb4024030 con 0x559391e90490 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.878+0000 7fdba6ffd640 1 -- 172.21.15.3:0/670435053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb401eb30 con 0x559391e90490 2024-09-12T02:21:07.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 -- 172.21.15.3:0/670435053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 msgr2=0x559391eb2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdbb401efd0 tx=0x7fdbb4009b60 comp rx=0 tx=0).stop 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 -- 172.21.15.3:0/670435053 shutdown_connections 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e90490 0x559391eb2620 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8fb70 0x559391e963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/670435053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e8c100 0x559391e86cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.880+0000 7fdbc5d37740 1 -- 172.21.15.3:0/670435053 >> 172.21.15.3:0/670435053 conn(0x559391e934d0 msgr2=0x559391e99b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.882+0000 7fdbc5d37740 1 -- 172.21.15.3:0/670435053 shutdown_connections 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.882+0000 7fdbc5d37740 1 -- 172.21.15.3:0/670435053 wait complete. 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.883+0000 7fdbc5d37740 1 -- start start 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.883+0000 7fdbc5d37740 1 Processor -- start 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e9e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1acf50 con 0x559391e8fb70 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1ac880 con 0x559391e90490 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.884+0000 7fdbc5d37740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc1a9a60 con 0x559391e8c100 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.885+0000 7fdbc16fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.885+0000 7fdbc16fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:51064/0 (socket says 172.21.15.3:51064) 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.885+0000 7fdbc16fa640 1 -- 172.21.15.3:0/928703534 learned_addr learned my addr 172.21.15.3:0/928703534 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.885+0000 7fdbc5d37740 10 client.0 _ll_register_callbacks cb 0x7fdbb8004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.885+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e9e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc16fa640 1 -- 172.21.15.3:0/928703534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 msgr2=0x559391e9e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc1efb640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc16fa640 1 -- 172.21.15.3:0/928703534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 msgr2=0x559391e9e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e9e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.886+0000 7fdbc16fa640 1 -- 172.21.15.3:0/928703534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559391e83260 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.890+0000 7fdbc16fa640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800bdf0 tx=0x7fdbb800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef75100f9f125abd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.890+0000 7fdba4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.893+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb801a670 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.893+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdbb801a9f0 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.893+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb4024030 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.894+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdbb801abf0 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.894+0000 7fdbc5d37740 10 client.25735 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.894+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559391f16930 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb8015070 con 0x559391e8c100 2024-09-12T02:21:07.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdbb8038050 con 0x559391e8c100 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdbc5d37740 10 client.25735 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdbc5d37740 10 client.25735 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdbc5d37740 20 client.25735 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391e9ad50 0x559391f19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.895+0000 7fdbc5d37740 4 client.25735 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.896+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x7fdbb4024030 con 0x559391e9ad50 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.897+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391e9ad50 0x559391f19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.897+0000 7fdbc0ef9640 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391e9ad50 0x559391f19050 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c0ef8c8c5c7cf5 server_cookie=3b127a2507d8fd96 in_seq=0 out_seq=0 2024-09-12T02:21:07.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:05.900+0000 7fdba4ff9640 10 client.25735 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:07.171 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:08.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:07 smithi026 ceph-mon[35496]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 125 MiB/s wr, 16.54k op/s 2024-09-12T02:21:08.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:07 smithi003 ceph-mon[31476]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 125 MiB/s wr, 16.54k op/s 2024-09-12T02:21:08.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:07 smithi092 ceph-mon[36089]: pgmap v418: 97 pgs: 97 active+clean; 13 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 125 MiB/s wr, 16.54k op/s 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.575+0000 7fba3e647640 1 -- 172.21.15.3:0/2365835623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38076230 msgr2=0x7fba38074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.575+0000 7fba3e647640 1 --2- 172.21.15.3:0/2365835623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38076230 0x7fba38074680 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fba3001acb0 tx=0x7fba3003b920 comp rx=0 tx=0).stop 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.576+0000 7fba3e647640 1 -- 172.21.15.3:0/2365835623 shutdown_connections 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.576+0000 7fba3e647640 1 --2- 172.21.15.3:0/2365835623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba3810e150 0x7fba38110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.576+0000 7fba3e647640 1 --2- 172.21.15.3:0/2365835623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba38074c50 0x7fba380750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.576+0000 7fba3e647640 1 --2- 172.21.15.3:0/2365835623 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38076230 0x7fba38074680 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.576+0000 7fba3e647640 1 -- 172.21.15.3:0/2365835623 >> 172.21.15.3:0/2365835623 conn(0x7fba380fd2e0 msgr2=0x7fba380ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.577+0000 7fba3e647640 1 -- 172.21.15.3:0/2365835623 shutdown_connections 2024-09-12T02:21:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.577+0000 7fba3e647640 1 -- 172.21.15.3:0/2365835623 wait complete. 2024-09-12T02:21:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.578+0000 7fba3e647640 1 Processor -- start 2024-09-12T02:21:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.578+0000 7fba3e647640 1 -- start start 2024-09-12T02:21:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38074c50 0x7fba38196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba38076230 0x7fba38196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba38108620 con 0x7fba38074c50 2024-09-12T02:21:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba381084a0 con 0x7fba38076230 2024-09-12T02:21:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3e647640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba381087a0 con 0x7fba3810e150 2024-09-12T02:21:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3cbe4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3cbe4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37690/0 (socket says 172.21.15.3:37690) 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba3cbe4640 1 -- 172.21.15.3:0/660646752 learned_addr learned my addr 172.21.15.3:0/660646752 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.579+0000 7fba377fe640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba38076230 0x7fba38196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 -- 172.21.15.3:0/660646752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba38076230 msgr2=0x7fba38196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba38076230 0x7fba38196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 -- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38074c50 msgr2=0x7fba38196110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38074c50 0x7fba38196110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 -- 172.21.15.3:0/660646752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3819d940 con 0x7fba3810e150 2024-09-12T02:21:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.580+0000 7fba3cbe4640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba2800d750 tx=0x7fba2800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.582+0000 7fba357fa640 1 -- 172.21.15.3:0/660646752 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28017070 con 0x7fba3810e150 2024-09-12T02:21:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.582+0000 7fba357fa640 1 -- 172.21.15.3:0/660646752 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba28005500 con 0x7fba3810e150 2024-09-12T02:21:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.582+0000 7fba357fa640 1 -- 172.21.15.3:0/660646752 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28004d90 con 0x7fba3810e150 2024-09-12T02:21:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.582+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3819dc30 con 0x7fba3810e150 2024-09-12T02:21:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.582+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3819e1b0 con 0x7fba3810e150 2024-09-12T02:21:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.586+0000 7fba413dd740 1 Processor -- start 2024-09-12T02:21:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.586+0000 7fba357fa640 1 -- 172.21.15.3:0/660646752 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fba28010460 con 0x7fba3810e150 2024-09-12T02:21:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 -- start start 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e52e0 0x562eea5e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5ea370 0x562eea5ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562eea5482f0 con 0x562eea5e49a0 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562eea4f0270 con 0x562eea5e52e0 2024-09-12T02:21:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.587+0000 7fba413dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562eea5442b0 con 0x562eea5ea370 2024-09-12T02:21:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.588+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.588+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44906/0 (socket says 172.21.15.3:44906) 2024-09-12T02:21:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.588+0000 7fba377fe640 1 -- 172.21.15.3:0/2316518322 learned_addr learned my addr 172.21.15.3:0/2316518322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.588+0000 7fba377fe640 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5ea370 0x562eea5ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.588+0000 7fba357fa640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 0x7fba200a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba357fa640 1 -- 172.21.15.3:0/660646752 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fba280cead0 con 0x7fba3810e150 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba377fe640 1 -- 172.21.15.3:0/2316518322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5ea370 msgr2=0x562eea5ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba377fe640 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5ea370 0x562eea5ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba377fe640 1 -- 172.21.15.3:0/2316518322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e52e0 msgr2=0x562eea5e9da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba377fe640 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e52e0 0x562eea5e9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.589+0000 7fba377fe640 1 -- 172.21.15.3:0/2316518322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eea4ef340 con 0x562eea5e49a0 2024-09-12T02:21:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.590+0000 7fba3cbe4640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 0x7fba200a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.590+0000 7fba377fe640 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba2c017680 tx=0x7fba2c009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdce524ff1c6919a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.591+0000 7fba1e7fc640 1 -- 172.21.15.3:0/2316518322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c013780 con 0x562eea5e49a0 2024-09-12T02:21:09.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.591+0000 7fba1e7fc640 1 -- 172.21.15.3:0/2316518322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2c01d070 con 0x562eea5e49a0 2024-09-12T02:21:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.592+0000 7fba413dd740 1 -- 172.21.15.3:0/2316518322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 msgr2=0x562eea5e4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.592+0000 7fba413dd740 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fba2c017680 tx=0x7fba2c009430 comp rx=0 tx=0).stop 2024-09-12T02:21:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.592+0000 7fba3cbe4640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 0x7fba200a5180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba28009610 tx=0x7fba280bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.593+0000 7fba413dd740 1 -- 172.21.15.3:0/2316518322 shutdown_connections 2024-09-12T02:21:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.593+0000 7fba413dd740 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5ea370 0x562eea5ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.593+0000 7fba413dd740 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e52e0 0x562eea5e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.593+0000 7fba413dd740 1 --2- 172.21.15.3:0/2316518322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5e4da0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.593+0000 7fba413dd740 1 -- 172.21.15.3:0/2316518322 >> 172.21.15.3:0/2316518322 conn(0x562eea5e0010 msgr2=0x562eea5e0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.603+0000 7fba413dd740 1 -- 172.21.15.3:0/2316518322 shutdown_connections 2024-09-12T02:21:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.604+0000 7fba413dd740 1 -- 172.21.15.3:0/2316518322 wait complete. 2024-09-12T02:21:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.605+0000 7fba413dd740 1 -- start start 2024-09-12T02:21:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.605+0000 7fba413dd740 1 Processor -- start 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.606+0000 7fba413dd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.606+0000 7fba413dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.606+0000 7fba413dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.607+0000 7fba413dd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562eea5ed330 con 0x562eea5e49a0 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.607+0000 7fba413dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562eea4d1290 con 0x562eea5ea370 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.607+0000 7fba413dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562eea5ed4b0 con 0x562eea5e52e0 2024-09-12T02:21:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.607+0000 7fba413dd740 10 client.0 _ll_register_callbacks cb 0x562eea502140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.608+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.608+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44908/0 (socket says 172.21.15.3:44908) 2024-09-12T02:21:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.608+0000 7fba377fe640 1 -- 172.21.15.3:0/1190343932 learned_addr learned my addr 172.21.15.3:0/1190343932 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.609+0000 7fba377fe640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba37fff640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba377fe640 1 -- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 msgr2=0x562eea5ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba377fe640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba377fe640 1 -- 172.21.15.3:0/1190343932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 msgr2=0x562eea5f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba377fe640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.610+0000 7fba377fe640 1 -- 172.21.15.3:0/1190343932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eea25b210 con 0x562eea5e49a0 2024-09-12T02:21:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.611+0000 7fba37fff640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.612+0000 7fba377fe640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0091e0 tx=0x7fba2c018470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7195f29d908076c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.612+0000 7fba1bff7640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.613+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c004230 con 0x562eea5e49a0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.613+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fba2c0043d0 con 0x562eea5e49a0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.614+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba2c01d070 con 0x562eea5e49a0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.614+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba2c0184e0 con 0x562eea5e49a0 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.614+0000 7fba413dd740 10 client.16140 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.615+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562eea327620 con 0x562eea5e49a0 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2c01d070 con 0x562eea5e49a0 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fba2c015050 con 0x562eea5e49a0 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 10 client.16140 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 20 client.16140 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea5fa4c0 0x562eea5fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 4 client.16140 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.620+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eea555a20 con 0x562eea5fa4c0 2024-09-12T02:21:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.622+0000 7fba37fff640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea5fa4c0 0x562eea5fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.626+0000 7fba37fff640 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea5fa4c0 0x562eea5fc8b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee153b4dafb987d server_cookie=46e1dc98912d1ce3 in_seq=0 out_seq=0 2024-09-12T02:21:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.626+0000 7fba1bff7640 10 client.16140 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.627+0000 7fba1bff7640 1 -- 172.21.15.3:0/1190343932 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eea555a20 con 0x562eea5fa4c0 2024-09-12T02:21:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.627+0000 7fba1bff7640 10 client.16140 handle_command_reply: tid=0 2024-09-12T02:21:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.627+0000 7fba413dd740 1 client.16140 shutdown 2024-09-12T02:21:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 msgr2=0x562eea5eeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0091e0 tx=0x7fba2c018470 comp rx=0 tx=0).stop 2024-09-12T02:21:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 shutdown_connections 2024-09-12T02:21:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea5fa4c0 0x562eea5fc8b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 --2- 172.21.15.3:0/1190343932 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e49a0 0x562eea5eeb80 secure :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0091e0 tx=0x7fba2c018470 comp rx=0 tx=0).stop 2024-09-12T02:21:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.628+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 >> 172.21.15.3:0/1190343932 conn(0x562eea5e0010 msgr2=0x562eea5e1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.630+0000 7fba1bff7640 0 client.16140 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.630+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 shutdown_connections 2024-09-12T02:21:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.631+0000 7fba413dd740 1 -- 172.21.15.3:0/1190343932 wait complete. 2024-09-12T02:21:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.631+0000 7fba413dd740 20 client.16140 trim_cache size 0 max 16384 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 Processor -- start 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 -- start start 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 0x562eea5e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400e610 con 0x562eea5ea370 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.666+0000 7fba413dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba24010240 con 0x562eea5e0ec0 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba413dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400ece0 con 0x562eea5e52e0 2024-09-12T02:21:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44918/0 (socket says 172.21.15.3:44918) 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 -- 172.21.15.3:0/3138149691 learned_addr learned my addr 172.21.15.3:0/3138149691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba377fe640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba377fe640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 0x562eea5e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 -- 172.21.15.3:0/3138149691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 msgr2=0x562eea5ef040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 -- 172.21.15.3:0/3138149691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 msgr2=0x562eea5e1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 0x562eea5e1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 -- 172.21.15.3:0/3138149691 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eea5edb60 con 0x562eea5ea370 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba377fe640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 0x562eea5e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba37fff640 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fba300407a0 tx=0x7fba30040870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e81f7905341e8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba1dffb640 1 -- 172.21.15.3:0/3138149691 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30045040 con 0x562eea5ea370 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba1dffb640 1 -- 172.21.15.3:0/3138149691 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba30057030 con 0x562eea5ea370 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.667+0000 7fba1dffb640 1 -- 172.21.15.3:0/3138149691 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30004280 con 0x562eea5ea370 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.668+0000 7fba413dd740 1 -- 172.21.15.3:0/3138149691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 msgr2=0x562eea5f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.668+0000 7fba413dd740 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fba300407a0 tx=0x7fba30040870 comp rx=0 tx=0).stop 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.668+0000 7fba413dd740 1 -- 172.21.15.3:0/3138149691 shutdown_connections 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5e0ec0 0x562eea5e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5ea370 0x562eea5f60c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- 172.21.15.3:0/3138149691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 -- 172.21.15.3:0/3138149691 >> 172.21.15.3:0/3138149691 conn(0x562eea5e0010 msgr2=0x562eea5ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 -- 172.21.15.3:0/3138149691 shutdown_connections 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 -- 172.21.15.3:0/3138149691 wait complete. 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 -- start start 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 Processor -- start 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.669+0000 7fba413dd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba413dd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400fae0 con 0x562eea5e0ec0 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba413dd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400f410 con 0x562eea5ea370 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba413dd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400c5f0 con 0x562eea5e52e0 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44932/0 (socket says 172.21.15.3:44932) 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 -- 172.21.15.3:0/2712957018 learned_addr learned my addr 172.21.15.3:0/2712957018 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba413dd740 10 client.0 _ll_register_callbacks cb 0x7fba2c009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba37fff640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 -- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 msgr2=0x562eea5f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 -- 172.21.15.3:0/2712957018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 msgr2=0x562eea5f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 -- 172.21.15.3:0/2712957018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eea5f57d0 con 0x562eea5e0ec0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba377fe640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x562eea5efa20 tx=0x7fba2c018470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3660af3db1a01d00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba1bff7640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c024420 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fba2c0094a0 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.670+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba30057030 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fba2c0184e0 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 10 client.16146 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562eea665630 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba30057030 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fba2c015040 con 0x562eea5e0ec0 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 10 client.16146 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 20 client.16146 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea666040 0x562eea668460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 4 client.16146 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.671+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562eea54fd70 con 0x562eea666040 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.672+0000 7fba37fff640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea666040 0x562eea668460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.672+0000 7fba37fff640 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea666040 0x562eea668460 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7ca93792a30c9c server_cookie=2dac690a6d976eaf in_seq=0 out_seq=0 2024-09-12T02:21:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:09.672+0000 7fba1bff7640 10 client.16146 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:10.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:09 smithi026 ceph-mon[35496]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 121 MiB/s wr, 16.16k op/s 2024-09-12T02:21:10.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:09 smithi003 ceph-mon[31476]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 121 MiB/s wr, 16.16k op/s 2024-09-12T02:21:10.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:09 smithi092 ceph-mon[36089]: pgmap v419: 97 pgs: 97 active+clean; 13 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 121 MiB/s wr, 16.16k op/s 2024-09-12T02:21:12.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:11 smithi026 ceph-mon[35496]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 148 MiB/s wr, 20.49k op/s 2024-09-12T02:21:12.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:11 smithi003 ceph-mon[31476]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 148 MiB/s wr, 20.49k op/s 2024-09-12T02:21:12.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:11 smithi092 ceph-mon[36089]: pgmap v420: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 148 MiB/s wr, 20.49k op/s 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba1bff7640 1 -- 172.21.15.3:0/2712957018 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562eea54fd70 con 0x562eea666040 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba1bff7640 10 client.16146 handle_command_reply: tid=0 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 client.16146 shutdown 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 msgr2=0x562eea5f3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x562eea5efa20 tx=0x7fba2c018470 comp rx=0 tx=0).stop 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 shutdown_connections 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x562eea666040 0x562eea668460 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562eea5ea370 0x562eea5f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562eea5e52e0 0x562eea5f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.210 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba1bff7640 0 client.16146 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 --2- 172.21.15.3:0/2712957018 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562eea5e0ec0 0x562eea5f3c70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 >> 172.21.15.3:0/2712957018 conn(0x562eea5e0010 msgr2=0x562eea5ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.209+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 shutdown_connections 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.210+0000 7fba413dd740 1 -- 172.21.15.3:0/2712957018 wait complete. 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.210+0000 7fba413dd740 20 client.16146 trim_cache size 0 max 16384 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 msgr2=0x7fba200a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 0x7fba200a5180 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fba28009610 tx=0x7fba280bb040 comp rx=0 tx=0).stop 2024-09-12T02:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 msgr2=0x7fba3819d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fba2800d750 tx=0x7fba2800dc10 comp rx=0 tx=0).stop 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 shutdown_connections 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fba200a2cc0 0x7fba200a5180 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba3810e150 0x7fba3819d2a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba38076230 0x7fba38196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 --2- 172.21.15.3:0/660646752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fba38074c50 0x7fba38196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 >> 172.21.15.3:0/660646752 conn(0x7fba380fd2e0 msgr2=0x7fba381013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 shutdown_connections 2024-09-12T02:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:12.211+0000 7fba3e647640 1 -- 172.21.15.3:0/660646752 wait complete. 2024-09-12T02:21:13.271 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:21:13.271 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"btime":"2024-09-12T02:19:58:341951+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:19:58.176754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:21:13.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.251+0000 7f152e81e640 1 -- 172.21.15.3:0/3997187527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806e4f0 msgr2=0x7f152806e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:13.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.251+0000 7f152e81e640 1 --2- 172.21.15.3:0/3997187527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806e4f0 0x7f152806e8d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1518009890 tx=0x7f151802a3b0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.254+0000 7f152e81e640 1 -- 172.21.15.3:0/3997187527 shutdown_connections 2024-09-12T02:21:13.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.254+0000 7f152e81e640 1 --2- 172.21.15.3:0/3997187527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 0x7f1528116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.254+0000 7f152e81e640 1 --2- 172.21.15.3:0/3997187527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806ee10 0x7f1528114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.254+0000 7f152e81e640 1 --2- 172.21.15.3:0/3997187527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806e4f0 0x7f152806e8d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.254+0000 7f152e81e640 1 -- 172.21.15.3:0/3997187527 >> 172.21.15.3:0/3997187527 conn(0x7f1528073180 msgr2=0x7f1528073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.259+0000 7f152e81e640 1 -- 172.21.15.3:0/3997187527 shutdown_connections 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.259+0000 7f152e81e640 1 -- 172.21.15.3:0/3997187527 wait complete. 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.260+0000 7f152e81e640 1 Processor -- start 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.261+0000 7f152e81e640 1 -- start start 2024-09-12T02:21:13.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806e4f0 0x7f15281a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 0x7f15281a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f152811a550 con 0x7f152806e4f0 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f152811a3d0 con 0x7f1528114ab0 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152e81e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f152811a6d0 con 0x7f152806ee10 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152d01b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152d01b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37740/0 (socket says 172.21.15.3:37740) 2024-09-12T02:21:13.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.262+0000 7f152d01b640 1 -- 172.21.15.3:0/3654141930 learned_addr learned my addr 172.21.15.3:0/3654141930 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.263+0000 7f152e01d640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 0x7f15281a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.263+0000 7f152d81c640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806e4f0 0x7f15281a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.265+0000 7f152d01b640 1 -- 172.21.15.3:0/3654141930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 msgr2=0x7f15281a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.265+0000 7f152d01b640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 0x7f15281a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.265+0000 7f152d01b640 1 -- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806e4f0 msgr2=0x7f15281a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.265+0000 7f152d01b640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806e4f0 0x7f15281a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.265+0000 7f152d01b640 1 -- 172.21.15.3:0/3654141930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281a0fc0 con 0x7f152806ee10 2024-09-12T02:21:13.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.270+0000 7f152d01b640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f152000db80 tx=0x7f152000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.271+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520017070 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.271+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1520003b70 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.271+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15200106c0 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.272+0000 7f152e81e640 1 -- 172.21.15.3:0/3654141930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15281a12b0 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.272+0000 7f152e81e640 1 -- 172.21.15.3:0/3654141930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152810a300 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.276+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152806e8d0 con 0x7f152806ee10 2024-09-12T02:21:13.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.281+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1520004020 con 0x7f152806ee10 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.282+0000 7f151effd640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 0x7f15140a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.282+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f15200ceac0 con 0x7f152806ee10 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.283+0000 7f152d81c640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 0x7f15140a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.283+0000 7f152d81c640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 0x7f15140a9620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1518005e00 tx=0x7f1518005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.291+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1520020070 con 0x7f152806ee10 2024-09-12T02:21:13.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.720+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15281ad5a0 con 0x7f152806ee10 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.721+0000 7f151effd640 1 -- 172.21.15.3:0/3654141930 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11954 (secure 0 0 0) 0x7f15200cee20 con 0x7f152806ee10 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 33 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 msgr2=0x7f15140a9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 0x7f15140a9620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1518005e00 tx=0x7f1518005d50 comp rx=0 tx=0).stop 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 msgr2=0x7f15281a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f152000db80 tx=0x7f152000b630 comp rx=0 tx=0).stop 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 shutdown_connections 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f15140a7160 0x7f15140a9620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1528114ab0 0x7f15281a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152806ee10 0x7f15281a6c10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 --2- 172.21.15.3:0/3654141930 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f152806e4f0 0x7f15281a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:13.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 >> 172.21.15.3:0/3654141930 conn(0x7f1528073180 msgr2=0x7f152806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:13.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.723+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 shutdown_connections 2024-09-12T02:21:13.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:12.724+0000 7f151cff9640 1 -- 172.21.15.3:0/3654141930 wait complete. 2024-09-12T02:21:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:13 smithi026 ceph-mon[35496]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 146 MiB/s wr, 21.43k op/s 2024-09-12T02:21:14.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:13 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3654141930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:13 smithi003 ceph-mon[31476]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 146 MiB/s wr, 21.43k op/s 2024-09-12T02:21:14.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:13 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3654141930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:14.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:13 smithi092 ceph-mon[36089]: pgmap v421: 97 pgs: 97 active+clean; 13 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 146 MiB/s wr, 21.43k op/s 2024-09-12T02:21:14.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:13 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3654141930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:16.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:15 smithi026 ceph-mon[35496]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 140 MiB/s wr, 20.89k op/s 2024-09-12T02:21:16.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:15 smithi003 ceph-mon[31476]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 140 MiB/s wr, 20.89k op/s 2024-09-12T02:21:16.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:15 smithi092 ceph-mon[36089]: pgmap v422: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 140 MiB/s wr, 20.89k op/s 2024-09-12T02:21:18.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:17 smithi026 ceph-mon[35496]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 145 MiB/s wr, 22.96k op/s 2024-09-12T02:21:18.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:17 smithi003 ceph-mon[31476]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 145 MiB/s wr, 22.96k op/s 2024-09-12T02:21:18.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:17 smithi092 ceph-mon[36089]: pgmap v423: 97 pgs: 97 active+clean; 13 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 145 MiB/s wr, 22.96k op/s 2024-09-12T02:21:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdba4ff9640 1 -- 172.21.15.3:0/928703534 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fdbb4024030 con 0x559391e9ad50 2024-09-12T02:21:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdba4ff9640 10 client.25735 handle_command_reply: tid=0 2024-09-12T02:21:19.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 client.25735 shutdown 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 msgr2=0x559391e9dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800bdf0 tx=0x7fdbb800b390 comp rx=0 tx=0).stop 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 shutdown_connections 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x559391e9ad50 0x559391f19050 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559391e90490 0x559391e9e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x559391e8fb70 0x559391e9e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 --2- 172.21.15.3:0/928703534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559391e8c100 0x559391e9dd80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 >> 172.21.15.3:0/928703534 conn(0x559391e934d0 msgr2=0x559391e91de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.569+0000 7fdba4ff9640 0 client.25735 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.570+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 shutdown_connections 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.570+0000 7fdbc5d37740 1 -- 172.21.15.3:0/928703534 wait complete. 2024-09-12T02:21:19.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.570+0000 7fdbc5d37740 20 client.25735 trim_cache size 0 max 16384 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 33, 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 6, 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "d6859252": { 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 6, 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:21:19.572 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 133.4, 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:19.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:19.574 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:19.574 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 msgr2=0x7fdba80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 0x7fdba80a53f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005fd0 tx=0x7fdbb4014b70 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 msgr2=0x7fdbbc19ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbac035040 tx=0x7fdbac033d10 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 shutdown_connections 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdba80a2f30 0x7fdba80a53f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbbc198660 0x7fdbbc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbbc1082a0 0x7fdbbc19ead0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 --2- 172.21.15.3:0/2899246520 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbbc102d30 0x7fdbbc19e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 >> 172.21.15.3:0/2899246520 conn(0x7fdbbc0fc7d0 msgr2=0x7fdbbc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:19.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 shutdown_connections 2024-09-12T02:21:19.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:19.571+0000 7fdbc395e640 1 -- 172.21.15.3:0/2899246520 wait complete. 2024-09-12T02:21:20.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:19 smithi026 ceph-mon[35496]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 136 MiB/s wr, 22.17k op/s 2024-09-12T02:21:20.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:19 smithi003 ceph-mon[31476]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 136 MiB/s wr, 22.17k op/s 2024-09-12T02:21:20.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:19 smithi092 ceph-mon[36089]: pgmap v424: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 136 MiB/s wr, 22.17k op/s 2024-09-12T02:21:22.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:21 smithi026 ceph-mon[35496]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 150 MiB/s wr, 25.02k op/s 2024-09-12T02:21:22.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:21 smithi003 ceph-mon[31476]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 150 MiB/s wr, 25.02k op/s 2024-09-12T02:21:22.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:21 smithi092 ceph-mon[36089]: pgmap v425: 97 pgs: 97 active+clean; 13 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 150 MiB/s wr, 25.02k op/s 2024-09-12T02:21:23.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:23 smithi003 ceph-mon[31476]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 115 MiB/s wr, 19.86k op/s 2024-09-12T02:21:24.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:23 smithi026 ceph-mon[35496]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 115 MiB/s wr, 19.86k op/s 2024-09-12T02:21:24.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:23 smithi092 ceph-mon[36089]: pgmap v426: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 115 MiB/s wr, 19.86k op/s 2024-09-12T02:21:24.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:21:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- 172.21.15.3:0/1817543729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8100a40 msgr2=0x7fa9e8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1817543729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8100a40 0x7fa9e8100e40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009900 tx=0x7fa9d802a410 comp rx=0 tx=0).stop 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- 172.21.15.3:0/1817543729 shutdown_connections 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1817543729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e81025f0 0x7fa9e8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1817543729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8101c30 0x7fa9e81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1817543729 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e8100a40 0x7fa9e8100e40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- 172.21.15.3:0/1817543729 >> 172.21.15.3:0/1817543729 conn(0x7fa9e80fc1c0 msgr2=0x7fa9e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- 172.21.15.3:0/1817543729 shutdown_connections 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- 172.21.15.3:0/1817543729 wait complete. 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 Processor -- start 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.310+0000 7fa9ee331640 1 -- start start 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8100a40 0x7fa9e8196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 0x7fa9e8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810c6c0 con 0x7fa9e81025f0 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810c540 con 0x7fa9e8101c30 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ee331640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810c840 con 0x7fa9e8100a40 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58846/0 (socket says 172.21.15.3:58846) 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1967956240 learned_addr learned my addr 172.21.15.3:0/1967956240 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8100a40 0x7fa9e8196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 0x7fa9e8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1967956240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8100a40 msgr2=0x7fa9e8196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8100a40 0x7fa9e8196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1967956240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 msgr2=0x7fa9e8197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 0x7fa9e8197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1967956240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e819e960 con 0x7fa9e81025f0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 0x7fa9e8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000cce0 tx=0x7fa9e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.311+0000 7fa9e57fa640 1 -- 172.21.15.3:0/1967956240 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e000f450 con 0x7fa9e81025f0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.312+0000 7fa9e57fa640 1 -- 172.21.15.3:0/1967956240 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e000fad0 con 0x7fa9e81025f0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.312+0000 7fa9e57fa640 1 -- 172.21.15.3:0/1967956240 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0018c10 con 0x7fa9e81025f0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.312+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e819ec50 con 0x7fa9e81025f0 2024-09-12T02:21:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.312+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e819f1a0 con 0x7fa9e81025f0 2024-09-12T02:21:25.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 Processor -- start 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9e57fa640 1 -- 172.21.15.3:0/1967956240 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa9e000f5f0 con 0x7fa9e81025f0 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 -- start start 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee2232f0 con 0x55e7ee2c5370 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee1cb270 con 0x55e7ee2c02e0 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee21f2b0 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9e77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58848/0 (socket says 172.21.15.3:58848) 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.314+0000 7fa9e77fe640 1 -- 172.21.15.3:0/888143641 learned_addr learned my addr 172.21.15.3:0/888143641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:25.322 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 -- 172.21.15.3:0/888143641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 msgr2=0x55e7ee2bfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 -- 172.21.15.3:0/888143641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 msgr2=0x55e7ee2c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 -- 172.21.15.3:0/888143641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ee1ca340 con 0x55e7ee2c5370 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e57fa640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 0x7fa9d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009460 tx=0x7fa9dc009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340bfdc9ae56316e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 0x7fa9d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9e57fa640 1 -- 172.21.15.3:0/1967956240 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9e00cf6c0 con 0x7fa9e81025f0 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9ce7fc640 1 -- 172.21.15.3:0/888143641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004030 con 0x55e7ee2c5370 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9ce7fc640 1 -- 172.21.15.3:0/888143641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc00fc40 con 0x55e7ee2c5370 2024-09-12T02:21:25.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.315+0000 7fa9ce7fc640 1 -- 172.21.15.3:0/888143641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0035f0 con 0x55e7ee2c5370 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- 172.21.15.3:0/888143641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 msgr2=0x55e7ee2c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009460 tx=0x7fa9dc009670 comp rx=0 tx=0).stop 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 0x7fa9d00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005ab0 tx=0x7fa9d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- 172.21.15.3:0/888143641 shutdown_connections 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2c7830 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/888143641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- 172.21.15.3:0/888143641 >> 172.21.15.3:0/888143641 conn(0x55e7ee2bb010 msgr2=0x55e7ee2bb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- 172.21.15.3:0/888143641 shutdown_connections 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- 172.21.15.3:0/888143641 wait complete. 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.316+0000 7fa9f10cd740 1 -- start start 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 Processor -- start 2024-09-12T02:21:25.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee2c8340 con 0x55e7ee2c5370 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee1ac290 con 0x55e7ee2c02e0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e7ee2c84c0 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9f10cd740 10 client.0 _ll_register_callbacks cb 0x55e7ee1dd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58864/0 (socket says 172.21.15.3:58864) 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1249865694 learned_addr learned my addr 172.21.15.3:0/1249865694 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 msgr2=0x55e7ee2c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1249865694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 msgr2=0x55e7ee2ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/1249865694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7edf36210 con 0x55e7ee2c5370 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa9e009bea0 tx=0x7fa9e009b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6158478253e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.317+0000 7fa9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00a02c0 con 0x55e7ee2c5370 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9e00a46f0 con 0x55e7ee2c5370 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9dc00fc40 con 0x55e7ee2c5370 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9e00a4b80 con 0x55e7ee2c5370 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 10 client.16164 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7ee002620 con 0x55e7ee2c5370 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc00fc40 con 0x55e7ee2c5370 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fa9e009b360 con 0x55e7ee2c5370 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 10 client.16164 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 20 client.16164 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee2d5910 0x55e7ee2d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 4 client.16164 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.318+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ee230a20 con 0x55e7ee2d5910 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.319+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee2d5910 0x55e7ee2d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.319+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee2d5910 0x55e7ee2d7d70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b820015d9a5ff1 server_cookie=7a87b1fedb50b6a4 in_seq=0 out_seq=0 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.319+0000 7fa9c77fe640 10 client.16164 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:25.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9c77fe640 1 -- 172.21.15.3:0/1249865694 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ee230a20 con 0x55e7ee2d5910 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9c77fe640 10 client.16164 handle_command_reply: tid=0 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 client.16164 shutdown 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 msgr2=0x55e7ee2d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa9e009bea0 tx=0x7fa9e009b870 comp rx=0 tx=0).stop 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 shutdown_connections 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee2d5910 0x55e7ee2d7d70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee2d1170 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9c77fe640 0 client.16164 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/1249865694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 >> 172.21.15.3:0/1249865694 conn(0x55e7ee2bb010 msgr2=0x55e7ee2bc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 shutdown_connections 2024-09-12T02:21:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 1 -- 172.21.15.3:0/1249865694 wait complete. 2024-09-12T02:21:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.320+0000 7fa9f10cd740 20 client.16164 trim_cache size 0 max 16384 2024-09-12T02:21:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.334+0000 7fa9f10cd740 1 Processor -- start 2024-09-12T02:21:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.334+0000 7fa9f10cd740 1 -- start start 2024-09-12T02:21:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.334+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.339 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 0x55e7ee2dc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a7eb0 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a9ae0 con 0x55e7ee2c5370 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a8580 con 0x55e7ee2c02e0 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58880/0 (socket says 172.21.15.3:58880) 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 -- 172.21.15.3:0/4041042124 learned_addr learned my addr 172.21.15.3:0/4041042124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 0x55e7ee2dc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 -- 172.21.15.3:0/4041042124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 msgr2=0x55e7ee2bc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2bc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 -- 172.21.15.3:0/4041042124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 msgr2=0x55e7ee2dc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 0x55e7ee2dc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 -- 172.21.15.3:0/4041042124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ee2c07c0 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 0x55e7ee2dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8101a90 tx=0x7fa9d803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98904f6d725e6630 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9cdffb640 1 -- 172.21.15.3:0/4041042124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8035040 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9cdffb640 1 -- 172.21.15.3:0/4041042124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8057070 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.335+0000 7fa9cdffb640 1 -- 172.21.15.3:0/4041042124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80035f0 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- 172.21.15.3:0/4041042124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 msgr2=0x55e7ee2bc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8101a90 tx=0x7fa9d803e070 comp rx=0 tx=0).stop 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- 172.21.15.3:0/4041042124 shutdown_connections 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c5370 0x55e7ee2dc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2c02e0 0x55e7ee2bc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/4041042124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee2bc400 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- 172.21.15.3:0/4041042124 >> 172.21.15.3:0/4041042124 conn(0x55e7ee2bb010 msgr2=0x55e7ee2ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- 172.21.15.3:0/4041042124 shutdown_connections 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- 172.21.15.3:0/4041042124 wait complete. 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 -- start start 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.336+0000 7fa9f10cd740 1 Processor -- start 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee33c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee33ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.342 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a9380 con 0x55e7ee2c5370 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a8cb0 con 0x55e7ee2c02e0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81a5e90 con 0x55e7ee2bf9a0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9f10cd740 10 client.0 _ll_register_callbacks cb 0x7fa9e00077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58890/0 (socket says 172.21.15.3:58890) 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/137178768 learned_addr learned my addr 172.21.15.3:0/137178768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee33c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee33ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 msgr2=0x55e7ee33c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee33c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/137178768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 msgr2=0x55e7ee33ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee33ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 -- 172.21.15.3:0/137178768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ee341bb0 con 0x55e7ee2c5370 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9e77fe640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee33ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9ec8ce640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00a4d40 tx=0x7fa9e009ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c130f917e7aa511 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.337+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e009c610 con 0x55e7ee2c5370 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9e009b360 con 0x55e7ee2c5370 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8057070 con 0x55e7ee2c5370 2024-09-12T02:21:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9e00a02c0 con 0x55e7ee2c5370 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 10 client.16170 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7ee342e30 con 0x55e7ee2c5370 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8057070 con 0x55e7ee2c5370 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fa9e00babe0 con 0x55e7ee2c5370 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 10 client.16170 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 20 client.16170 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee3436d0 0x55e7ee345af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 4 client.16170 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7ee2c50c0 con 0x55e7ee3436d0 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.338+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee3436d0 0x55e7ee345af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.339+0000 7fa9c77fe640 10 client.16170 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:25.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.339+0000 7fa9e7fff640 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee3436d0 0x55e7ee345af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee72072946a41538 server_cookie=7446fe3a709f2bed in_seq=0 out_seq=0 2024-09-12T02:21:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9c77fe640 1 -- 172.21.15.3:0/137178768 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7ee2c50c0 con 0x55e7ee3436d0 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9c77fe640 10 client.16170 handle_command_reply: tid=0 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 client.16170 shutdown 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 msgr2=0x55e7ee341380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00a4d40 tx=0x7fa9e009ae50 comp rx=0 tx=0).stop 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 shutdown_connections 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e7ee3436d0 0x55e7ee345af0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e7ee2c5370 0x55e7ee341380 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e7ee2c02e0 0x55e7ee33ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 --2- 172.21.15.3:0/137178768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e7ee2bf9a0 0x55e7ee33c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9c77fe640 0 client.16170 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 >> 172.21.15.3:0/137178768 conn(0x55e7ee2bb010 msgr2=0x55e7ee2ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 shutdown_connections 2024-09-12T02:21:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 1 -- 172.21.15.3:0/137178768 wait complete. 2024-09-12T02:21:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.352+0000 7fa9f10cd740 20 client.16170 trim_cache size 0 max 16384 2024-09-12T02:21:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:21:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:21:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:21:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 msgr2=0x7fa9d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 0x7fa9d00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005ab0 tx=0x7fa9d8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 msgr2=0x7fa9e819e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000cce0 tx=0x7fa9e000a620 comp rx=0 tx=0).stop 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 shutdown_connections 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9d00a2c00 0x7fa9d00a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e81025f0 0x7fa9e819e2f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e8101c30 0x7fa9e8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 --2- 172.21.15.3:0/1967956240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e8100a40 0x7fa9e8196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 >> 172.21.15.3:0/1967956240 conn(0x7fa9e80fc1c0 msgr2=0x7fa9e80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 shutdown_connections 2024-09-12T02:21:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:25.353+0000 7fa9ee331640 1 -- 172.21.15.3:0/1967956240 wait complete. 2024-09-12T02:21:26.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:25 smithi026 ceph-mon[35496]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 88 MiB/s wr, 15.26k op/s 2024-09-12T02:21:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:25 smithi003 ceph-mon[31476]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 88 MiB/s wr, 15.26k op/s 2024-09-12T02:21:26.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:25 smithi092 ceph-mon[36089]: pgmap v427: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 88 MiB/s wr, 15.26k op/s 2024-09-12T02:21:27.935 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:27 smithi003 ceph-mon[31476]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 91 MiB/s wr, 15.48k op/s 2024-09-12T02:21:27.968 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd6859252', quiesce took 65.3 seconds. Will release after: -51.9 2024-09-12T02:21:27.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:21:28.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:27 smithi026 ceph-mon[35496]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 91 MiB/s wr, 15.48k op/s 2024-09-12T02:21:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:27 smithi092 ceph-mon[36089]: pgmap v428: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 91 MiB/s wr, 15.48k op/s 2024-09-12T02:21:28.090 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:28.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.720+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1578933250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 msgr2=0x7f4bcc1124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.720+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1578933250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1124e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009900 tx=0x7f4bbc02a410 comp rx=0 tx=0).stop 2024-09-12T02:21:28.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.723+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1578933250 shutdown_connections 2024-09-12T02:21:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.723+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1578933250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bcc112a20 0x7f4bcc114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.723+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1578933250 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1124e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.723+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1578933250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc076720 0x7f4bcc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.723+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1578933250 >> 172.21.15.3:0/1578933250 conn(0x7f4bcc0ffa80 msgr2=0x7f4bcc101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.724+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1578933250 shutdown_connections 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.724+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1578933250 wait complete. 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.725+0000 7f4bd1a3a640 1 Processor -- start 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.725+0000 7f4bd1a3a640 1 -- start start 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.725+0000 7f4bd1a3a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bcc076720 0x7f4bcc107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.725+0000 7f4bd1a3a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bd1a3a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc112a20 0x7f4bcc072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bd1a3a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc118630 con 0x7f4bcc077040 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bd1a3a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1184b0 con 0x7f4bcc112a20 2024-09-12T02:21:28.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bd1a3a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bcc1187b0 con 0x7f4bcc076720 2024-09-12T02:21:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53066/0 (socket says 172.21.15.3:53066) 2024-09-12T02:21:28.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.726+0000 7f4bca7fc640 1 -- 172.21.15.3:0/1721130175 learned_addr learned my addr 172.21.15.3:0/1721130175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.728+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc112a20 0x7f4bcc072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.733+0000 7f4bca7fc640 1 -- 172.21.15.3:0/1721130175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bcc076720 msgr2=0x7f4bcc107470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.733+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bcc076720 0x7f4bcc107470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.733+0000 7f4bca7fc640 1 -- 172.21.15.3:0/1721130175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc112a20 msgr2=0x7f4bcc072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.733+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc112a20 0x7f4bcc072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.733+0000 7f4bca7fc640 1 -- 172.21.15.3:0/1721130175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bcc073770 con 0x7f4bcc077040 2024-09-12T02:21:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.734+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009850 tx=0x7f4bbc0309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.736+0000 7f4bd0a38640 1 -- 172.21.15.3:0/1721130175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc03f070 con 0x7f4bcc077040 2024-09-12T02:21:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.736+0000 7f4bd0a38640 1 -- 172.21.15.3:0/1721130175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc0035f0 con 0x7f4bcc077040 2024-09-12T02:21:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.737+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bcc073a00 con 0x7f4bcc077040 2024-09-12T02:21:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.737+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bcc10f3a0 con 0x7f4bcc077040 2024-09-12T02:21:28.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.737+0000 7f4bd0a38640 1 -- 172.21.15.3:0/1721130175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc035030 con 0x7f4bcc077040 2024-09-12T02:21:28.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.741+0000 7f4bd0a38640 1 -- 172.21.15.3:0/1721130175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4bbc042070 con 0x7f4bcc077040 2024-09-12T02:21:28.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.745+0000 7f4bd0a38640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 0x7f4bb40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.745+0000 7f4bd0a38640 1 -- 172.21.15.3:0/1721130175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4bbc0efcc0 con 0x7f4bcc077040 2024-09-12T02:21:28.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.745+0000 7f4bd3e13740 1 Processor -- start 2024-09-12T02:21:28.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.747+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 0x7f4bb40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.750+0000 7f4bd3e13740 1 -- start start 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2737352f0 con 0x55a2737d0b40 2024-09-12T02:21:28.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2736dccf0 con 0x55a2737da370 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2737312b0 con 0x55a2737cf5d0 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 0x7f4bb40a5190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005fd0 tx=0x7f4bc40096f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53074/0 (socket says 172.21.15.3:53074) 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.751+0000 7f4bcaffd640 1 -- 172.21.15.3:0/2081219658 learned_addr learned my addr 172.21.15.3:0/2081219658 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:28.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.752+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.754+0000 7f4bcaffd640 1 -- 172.21.15.3:0/2081219658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 msgr2=0x55a2737cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.754+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.754+0000 7f4bcaffd640 1 -- 172.21.15.3:0/2081219658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 msgr2=0x55a2737dc760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:21:28.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.754+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737dc760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.754+0000 7f4bcaffd640 1 -- 172.21.15.3:0/2081219658 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2736c13a0 con 0x55a2737d0b40 2024-09-12T02:21:28.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.755+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009b10 tx=0x7f4bc4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4352f4c8e9676e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.757+0000 7f4bb1ffb640 1 -- 172.21.15.3:0/2081219658 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc401d640 con 0x55a2737d0b40 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.757+0000 7f4bb1ffb640 1 -- 172.21.15.3:0/2081219658 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc4022070 con 0x55a2737d0b40 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2081219658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 msgr2=0x55a2737cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009b10 tx=0x7f4bc4010000 comp rx=0 tx=0).stop 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2081219658 shutdown_connections 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2081219658 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d0b40 0x55a2737cf090 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009b10 tx=0x7f4bc4010000 comp rx=0 tx=0).stop 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.758+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2081219658 >> 172.21.15.3:0/2081219658 conn(0x55a273798310 msgr2=0x55a2737cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.759+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2081219658 shutdown_connections 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.759+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2081219658 wait complete. 2024-09-12T02:21:28.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.759+0000 7f4bd3e13740 1 -- start start 2024-09-12T02:21:28.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 Processor -- start 2024-09-12T02:21:28.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737cf5d0 0x55a2737d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737da370 0x55a2737d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a2737dd060 con 0x55a2737da370 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55a2736f0060 con 0x55a2737cf5d0 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55a2737dd1e0 con 0x55a2737d7bb0 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.760+0000 7f4bd3e13740 10 client.0 _ll_register_callbacks cb 0x55a2734438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:28.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.761+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.762+0000 7f4bcb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48226/0 (socket says 172.21.15.3:48226) 2024-09-12T02:21:28.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.762+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/2045455870 learned_addr learned my addr 172.21.15.3:0/2045455870 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:28.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.762+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737da370 0x55a2737d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.765 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.762+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737cf5d0 0x55a2737d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.763+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/2045455870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737cf5d0 msgr2=0x55a2737d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.763+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737cf5d0 0x55a2737d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.763+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737da370 msgr2=0x55a2737d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.763+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737da370 0x55a2737d7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.763+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/2045455870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2734a63c0 con 0x55a2737d7bb0 2024-09-12T02:21:28.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.764+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000e250 tx=0x7f4bc000e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b643b1b55384071d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.764+0000 7f4baaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:28.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.766+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0017070 con 0x55a2737d7bb0 2024-09-12T02:21:28.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.767+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4bc0013680 con 0x55a2737d7bb0 2024-09-12T02:21:28.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.771+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bc4022070 con 0x55a2737d7bb0 2024-09-12T02:21:28.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.778+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4bc0021050 con 0x55a2737d7bb0 2024-09-12T02:21:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.782+0000 7f4bd3e13740 10 client.25774 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:28.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.782+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2737e9070 con 0x55a2737d7bb0 2024-09-12T02:21:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.787+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc4022070 con 0x55a2737d7bb0 2024-09-12T02:21:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.787+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f4bc001ca60 con 0x55a2737d7bb0 2024-09-12T02:21:28.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 10 client.25774 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 10 client.25774 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 20 client.25774 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2737ea580 0x55a2737ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 4 client.25774 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.788+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a273742860 con 0x55a2737ea580 2024-09-12T02:21:28.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.789+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2737ea580 0x55a2737ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.790+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2737ea580 0x55a2737ec9c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71e2e15d84e9ba97 server_cookie=ab518769bddeddea in_seq=0 out_seq=0 2024-09-12T02:21:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.790+0000 7f4baaffd640 10 client.25774 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:21:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.791+0000 7f4baaffd640 1 -- 172.21.15.3:0/2045455870 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a273742860 con 0x55a2737ea580 2024-09-12T02:21:28.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.791+0000 7f4baaffd640 10 client.25774 handle_command_reply: tid=0 2024-09-12T02:21:28.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.793+0000 7f4bd3e13740 1 client.25774 shutdown 2024-09-12T02:21:28.794 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.796+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 msgr2=0x55a2737e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.796+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000e250 tx=0x7f4bc000e710 comp rx=0 tx=0).stop 2024-09-12T02:21:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 shutdown_connections 2024-09-12T02:21:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2737ea580 0x55a2737ec9c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737d7bb0 0x55a2737e65c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737da370 0x55a2737d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/2045455870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737cf5d0 0x55a2737d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 >> 172.21.15.3:0/2045455870 conn(0x55a273798310 msgr2=0x55a2737d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.797+0000 7f4baaffd640 0 client.25774 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:21:28.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.801+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 shutdown_connections 2024-09-12T02:21:28.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.803+0000 7f4bd3e13740 1 -- 172.21.15.3:0/2045455870 wait complete. 2024-09-12T02:21:28.804 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.804+0000 7f4bd3e13740 20 client.25774 trim_cache size 0 max 16384 2024-09-12T02:21:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.834+0000 7f4bd3e13740 1 Processor -- start 2024-09-12T02:21:28.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.834+0000 7f4bd3e13740 1 -- start start 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.834+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737cf5d0 0x55a2737e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 0x55a2737e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 0x55a2737f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb800e5f0 con 0x55a2737cf5d0 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 0x55a2737e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bca7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 0x55a2737e68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38334/0 (socket says 172.21.15.3:38334) 2024-09-12T02:21:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bca7fc640 1 -- 172.21.15.3:0/758044918 learned_addr learned my addr 172.21.15.3:0/758044918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 0x55a2737f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8010220 con 0x55a2737d7bb0 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.835+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb800ecc0 con 0x55a2737da370 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737cf5d0 0x55a2737e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/758044918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 msgr2=0x55a2737e68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 0x55a2737e68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/758044918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737cf5d0 msgr2=0x55a2737e63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737cf5d0 0x55a2737e63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.836+0000 7f4bcb7fe640 1 -- 172.21.15.3:0/758044918 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2737cf2d0 con 0x55a2737da370 2024-09-12T02:21:28.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.837+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 0x55a2737f2290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001f70 tx=0x7f4bc0012620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67c617c45d959ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.838+0000 7f4bb17fa640 1 -- 172.21.15.3:0/758044918 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc002b070 con 0x55a2737da370 2024-09-12T02:21:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.838+0000 7f4bb17fa640 1 -- 172.21.15.3:0/758044918 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0006c70 con 0x55a2737da370 2024-09-12T02:21:28.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.839+0000 7f4bb17fa640 1 -- 172.21.15.3:0/758044918 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0017070 con 0x55a2737da370 2024-09-12T02:21:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.839+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 msgr2=0x55a2737f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.840+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 0x55a2737f2290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001f70 tx=0x7f4bc0012620 comp rx=0 tx=0).stop 2024-09-12T02:21:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 shutdown_connections 2024-09-12T02:21:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737da370 0x55a2737f2290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737d7bb0 0x55a2737e68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/758044918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737cf5d0 0x55a2737e63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 >> 172.21.15.3:0/758044918 conn(0x55a273798310 msgr2=0x55a2737d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 shutdown_connections 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.841+0000 7f4bd3e13740 1 -- 172.21.15.3:0/758044918 wait complete. 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 -- start start 2024-09-12T02:21:28.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 Processor -- start 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d7bb0 0x55a2737d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb800fac0 con 0x55a2737d7bb0 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb800f3f0 con 0x55a2737da370 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.842+0000 7f4bd3e13740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb800c5d0 con 0x55a2737cf5d0 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.843+0000 7f4bd3e13740 10 client.0 _ll_register_callbacks cb 0x7f4bc0009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.843+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48246/0 (socket says 172.21.15.3:48246) 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 -- 172.21.15.3:0/3609551407 learned_addr learned my addr 172.21.15.3:0/3609551407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 -- 172.21.15.3:0/3609551407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 msgr2=0x55a2737d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737d42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 -- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d7bb0 msgr2=0x55a2737d3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d7bb0 0x55a2737d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.845+0000 7f4bcaffd640 1 -- 172.21.15.3:0/3609551407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2737d49b0 con 0x55a2737cf5d0 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.846+0000 7f4bcb7fe640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.846+0000 7f4bcaffd640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4010d10 tx=0x7f4bc400dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5b8d937febc4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:28.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.846+0000 7f4baaffd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.846+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400fe40 con 0x55a2737cf5d0 2024-09-12T02:21:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.846+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4bc4004a00 con 0x55a2737cf5d0 2024-09-12T02:21:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.847+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bc0006c70 con 0x55a2737cf5d0 2024-09-12T02:21:28.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.849+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4bc4004c00 con 0x55a2737cf5d0 2024-09-12T02:21:28.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.851+0000 7f4bd3e13740 10 client.25780 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:28.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.851+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a273855160 con 0x55a2737cf5d0 2024-09-12T02:21:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0006c70 con 0x55a2737cf5d0 2024-09-12T02:21:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f4bc4004460 con 0x55a2737cf5d0 2024-09-12T02:21:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 10 client.25780 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 10 client.25780 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-12T02:21:28.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 20 client.25780 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:28.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2738558b0 0x55a273857cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:28.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 4 client.25780 mds_command: new command op to 14619 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:21:28.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 --> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a27373cd70 con 0x55a2738558b0 2024-09-12T02:21:28.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.855+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2738558b0 0x55a273857cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:28.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.858+0000 7f4bca7fc640 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2738558b0 0x55a273857cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=882f4d1592209ff3 server_cookie=9fd2d9471824d4a7 in_seq=0 out_seq=0 2024-09-12T02:21:28.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.858+0000 7f4baaffd640 10 client.25780 ms_handle_connect on v2:172.21.15.3:6840/3157578506 2024-09-12T02:21:28.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.864+0000 7f4baaffd640 1 -- 172.21.15.3:0/3609551407 <== mds.4 v2:172.21.15.3:6840/3157578506 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a27373cd70 con 0x55a2738558b0 2024-09-12T02:21:28.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.864+0000 7f4baaffd640 10 client.25780 handle_command_reply: tid=0 2024-09-12T02:21:28.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.864+0000 7f4bd3e13740 1 client.25780 shutdown 2024-09-12T02:21:28.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 msgr2=0x55a2737e2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4010d10 tx=0x7f4bc400dbc0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 shutdown_connections 2024-09-12T02:21:28.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2737da370 0x55a2737d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a2737d7bb0 0x55a2737d3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.3:6840/3157578506,v1:172.21.15.3:6841/3157578506] conn(0x55a2738558b0 0x55a273857cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 --2- 172.21.15.3:0/3609551407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a2737cf5d0 0x55a2737e2b60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 >> 172.21.15.3:0/3609551407 conn(0x55a273798310 msgr2=0x55a2737d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4baaffd640 0 client.25780 ms_handle_reset on v2:172.21.15.3:6840/3157578506 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 shutdown_connections 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 1 -- 172.21.15.3:0/3609551407 wait complete. 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.868+0000 7f4bd3e13740 20 client.25780 trim_cache size 0 max 16384 2024-09-12T02:21:28.870 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 msgr2=0x7f4bb40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 0x7f4bb40a5190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005fd0 tx=0x7f4bc40096f0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 msgr2=0x7f4bcc1079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009850 tx=0x7f4bbc0309c0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 shutdown_connections 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f4bb40a2cd0 0x7f4bb40a5190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bcc112a20 0x7f4bcc072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4bcc077040 0x7f4bcc1079b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 --2- 172.21.15.3:0/1721130175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4bcc076720 0x7f4bcc107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:28.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 >> 172.21.15.3:0/1721130175 conn(0x7f4bcc0ffa80 msgr2=0x7f4bcc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:28.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 shutdown_connections 2024-09-12T02:21:28.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:28.870+0000 7f4bd1a3a640 1 -- 172.21.15.3:0/1721130175 wait complete. 2024-09-12T02:21:29.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:29 smithi003 ceph-mon[31476]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 10.38k op/s 2024-09-12T02:21:29.861 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:21:30.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:29 smithi026 ceph-mon[35496]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 10.38k op/s 2024-09-12T02:21:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:29 smithi092 ceph-mon[36089]: pgmap v429: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 62 MiB/s wr, 10.38k op/s 2024-09-12T02:21:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 -- 172.21.15.3:0/4065572304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 msgr2=0x7fbebc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:30.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 --2- 172.21.15.3:0/4065572304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc100ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009930 tx=0x7fbeac02a420 comp rx=0 tx=0).stop 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 -- 172.21.15.3:0/4065572304 shutdown_connections 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 --2- 172.21.15.3:0/4065572304 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 0x7fbebc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 --2- 172.21.15.3:0/4065572304 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc100ee0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 --2- 172.21.15.3:0/4065572304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 0x7fbebc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 -- 172.21.15.3:0/4065572304 >> 172.21.15.3:0/4065572304 conn(0x7fbebc0fc7a0 msgr2=0x7fbebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.101+0000 7fbec4371640 1 -- 172.21.15.3:0/4065572304 shutdown_connections 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.102+0000 7fbec4371640 1 -- 172.21.15.3:0/4065572304 wait complete. 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.102+0000 7fbec4371640 1 Processor -- start 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.102+0000 7fbec4371640 1 -- start start 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 0x7fbebc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:30.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 0x7fbebc19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc110e80 con 0x7fbebc100a80 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc110d00 con 0x7fbebc106a80 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec4371640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc111000 con 0x7fbebc101420 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53148/0 (socket says 172.21.15.3:53148) 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 -- 172.21.15.3:0/868243978 learned_addr learned my addr 172.21.15.3:0/868243978 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec190c640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 0x7fbebc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec290e640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 0x7fbebc19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 -- 172.21.15.3:0/868243978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 msgr2=0x7fbebc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:30.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 0x7fbebc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 -- 172.21.15.3:0/868243978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 msgr2=0x7fbebc19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 0x7fbebc19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 -- 172.21.15.3:0/868243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc19c270 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.103+0000 7fbec210d640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c920 tx=0x7fbeb800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.104+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8007c70 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.104+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb8007e10 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.104+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc19c560 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.104+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8018890 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.104+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc06a790 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.105+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc100ee0 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.106+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbeb8018a50 con 0x7fbebc100a80 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.107+0000 7fbeb37fe640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 0x7fbea80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:30.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.107+0000 7fbec190c640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 0x7fbea80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.107+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbeb80cf360 con 0x7fbebc100a80 2024-09-12T02:21:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.107+0000 7fbec190c640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 0x7fbea80a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:30.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.110+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbeb8014030 con 0x7fbebc100a80 2024-09-12T02:21:30.198 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:30.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.330+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbebc198620 con 0x7fbebc100a80 2024-09-12T02:21:30.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.331+0000 7fbeb37fe640 1 -- 172.21.15.3:0/868243978 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11954 (secure 0 0 0) 0x7fbeb809c3c0 con 0x7fbebc100a80 2024-09-12T02:21:30.332 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:21:30.333 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":33,"btime":"2024-09-12T02:19:58:341951+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:19:58.176754+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:21:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 33 2024-09-12T02:21:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.333+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 msgr2=0x7fbea80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.333+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 0x7fbea80a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac0058d0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.333+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 msgr2=0x7fbebc1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.333+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800c920 tx=0x7fbeb800cde0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 shutdown_connections 2024-09-12T02:21:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbea80a2c00 0x7fbea80a50c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc106a80 0x7fbebc19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbebc101420 0x7fbebc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 --2- 172.21.15.3:0/868243978 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbebc100a80 0x7fbebc1972c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 >> 172.21.15.3:0/868243978 conn(0x7fbebc0fc7a0 msgr2=0x7fbebc0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 shutdown_connections 2024-09-12T02:21:30.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:30.334+0000 7fbec4371640 1 -- 172.21.15.3:0/868243978 wait complete. 2024-09-12T02:21:31.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:30 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/868243978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:31.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:30 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/868243978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:31.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:30 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/868243978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:21:32.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:31 smithi026 ceph-mon[35496]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 65 MiB/s wr, 10.78k op/s 2024-09-12T02:21:32.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:31 smithi003 ceph-mon[31476]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 65 MiB/s wr, 10.78k op/s 2024-09-12T02:21:32.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:31 smithi092 ceph-mon[36089]: pgmap v430: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 65 MiB/s wr, 10.78k op/s 2024-09-12T02:21:34.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:33 smithi026 ceph-mon[35496]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 6.49k op/s 2024-09-12T02:21:34.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:33 smithi003 ceph-mon[31476]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 6.49k op/s 2024-09-12T02:21:34.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:33 smithi092 ceph-mon[36089]: pgmap v431: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 41 MiB/s wr, 6.49k op/s 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.397+0000 7fefad59a640 1 -- 172.21.15.3:0/2852345062 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4040860 con 0x7fefb0109830 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 -- 172.21.15.3:0/2852345062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 msgr2=0x7fefb0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2852345062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb0109c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fefa401a910 tx=0x7fefa403b430 comp rx=0 tx=0).stop 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 -- 172.21.15.3:0/2852345062 shutdown_connections 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2852345062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 0x7fefb011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2852345062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb0109c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2852345062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.398+0000 7fefb4c79640 1 -- 172.21.15.3:0/2852345062 >> 172.21.15.3:0/2852345062 conn(0x7fefb006e140 msgr2=0x7fefb006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.399+0000 7fefb4c79640 1 -- 172.21.15.3:0/2852345062 shutdown_connections 2024-09-12T02:21:35.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.399+0000 7fefb4c79640 1 -- 172.21.15.3:0/2852345062 wait complete. 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 Processor -- start 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 -- start start 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb011a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 0x7fefb0116d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0113300 con 0x7fefb011b140 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0113180 con 0x7fefb0109450 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.400+0000 7fefb4c79640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0113480 con 0x7fefb0109830 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.401+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.401+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48284/0 (socket says 172.21.15.3:48284) 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.401+0000 7fefadd9b640 1 -- 172.21.15.3:0/2684015678 learned_addr learned my addr 172.21.15.3:0/2684015678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.402+0000 7fefaed9d640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 0x7fefb0116d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.402+0000 7fefae59c640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb011a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 -- 172.21.15.3:0/2684015678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 msgr2=0x7fefb011a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb011a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 -- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 msgr2=0x7fefb0116d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 0x7fefb0116d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 -- 172.21.15.3:0/2684015678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb0117240 con 0x7fefb0109830 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefae59c640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb011a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fefadd9b640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa403b950 tx=0x7fefa4040d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.403+0000 7fef9f7fe640 1 -- 172.21.15.3:0/2684015678 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa404f070 con 0x7fefb0109830 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.404+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb01174d0 con 0x7fefb0109830 2024-09-12T02:21:35.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.404+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb01ab390 con 0x7fefb0109830 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.404+0000 7fef9f7fe640 1 -- 172.21.15.3:0/2684015678 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4004ba0 con 0x7fefb0109830 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.404+0000 7fef9f7fe640 1 -- 172.21.15.3:0/2684015678 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4046030 con 0x7fefb0109830 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.408+0000 7fefb7052740 1 Processor -- start 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.408+0000 7fef9f7fe640 1 -- 172.21.15.3:0/2684015678 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fefa4052070 con 0x7fefb0109830 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.410+0000 7fef9f7fe640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 0x7fef980a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.410+0000 7fef9f7fe640 1 -- 172.21.15.3:0/2684015678 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fefa41000e0 con 0x7fefb0109830 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.410+0000 7fefaed9d640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 0x7fef980a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.410+0000 7fefb7052740 1 -- start start 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.411+0000 7fefaed9d640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 0x7fef980a5210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005fd0 tx=0x7fefa000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.411+0000 7fefb7052740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bf4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.411+0000 7fefb7052740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.411+0000 7fefb7052740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefb7052740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5651c5b5a2f0 con 0x5651c5bff370 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefb7052740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5651c5b01cf0 con 0x5651c5bf45d0 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefb7052740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5651c5b562b0 con 0x5651c5bf5b40 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38390/0 (socket says 172.21.15.3:38390) 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 -- 172.21.15.3:0/19628589 learned_addr learned my addr 172.21.15.3:0/19628589 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefae59c640 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bf4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefaed9d640 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 -- 172.21.15.3:0/19628589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 msgr2=0x5651c5bf4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bf4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 -- 172.21.15.3:0/19628589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 msgr2=0x5651c5c01760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c01760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.412+0000 7fefadd9b640 1 -- 172.21.15.3:0/19628589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c5ae63a0 con 0x5651c5bf45d0 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.413+0000 7fefadd9b640 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefa40cc860 tx=0x7fefa40cc760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a25d8b0ccf8f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.413+0000 7fef9cff9640 1 -- 172.21.15.3:0/19628589 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4059040 con 0x5651c5bf45d0 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.413+0000 7fef9cff9640 1 -- 172.21.15.3:0/19628589 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa40e5070 con 0x5651c5bf45d0 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.413+0000 7fef9cff9640 1 -- 172.21.15.3:0/19628589 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa40d50b0 con 0x5651c5bf45d0 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 -- 172.21.15.3:0/19628589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 msgr2=0x5651c5bf4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefa40cc860 tx=0x7fefa40cc760 comp rx=0 tx=0).stop 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 -- 172.21.15.3:0/19628589 shutdown_connections 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c01760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bf4a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 --2- 172.21.15.3:0/19628589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bf4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.414+0000 7fefb7052740 1 -- 172.21.15.3:0/19628589 >> 172.21.15.3:0/19628589 conn(0x5651c5bbd310 msgr2=0x5651c5bf0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.415+0000 7fefb7052740 1 -- 172.21.15.3:0/19628589 shutdown_connections 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.416+0000 7fefb7052740 1 -- 172.21.15.3:0/19628589 wait complete. 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.417+0000 7fefb7052740 1 -- start start 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.417+0000 7fefb7052740 1 Processor -- start 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bfed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5bfc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5651c5c01e30 con 0x5651c5bff370 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5651c5b15060 con 0x5651c5bf45d0 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5651c5c01fb0 con 0x5651c5bf5b40 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefb7052740 10 client.0 _ll_register_callbacks cb 0x5651c58688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bfed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48308/0 (socket says 172.21.15.3:48308) 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefadd9b640 1 -- 172.21.15.3:0/3350017726 learned_addr learned my addr 172.21.15.3:0/3350017726 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.419+0000 7fefaed9d640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5bfc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefadd9b640 1 -- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 msgr2=0x5651c5bfed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefadd9b640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bfed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefadd9b640 1 -- 172.21.15.3:0/3350017726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 msgr2=0x5651c5bfc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefadd9b640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5bfc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefadd9b640 1 -- 172.21.15.3:0/3350017726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c58cb3c0 con 0x5651c5bf5b40 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.420+0000 7fefaed9d640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5bfc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.421+0000 7fefadd9b640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ec0 tx=0x7fefa401a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a66ea702bb96de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.421+0000 7fef95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.421+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa40da060 con 0x5651c5bf5b40 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.421+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fefa40eb210 con 0x5651c5bf5b40 2024-09-12T02:21:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.422+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa40e5070 con 0x5651c5bf5b40 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.422+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fefa40eb070 con 0x5651c5bf5b40 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.422+0000 7fefb7052740 10 client.25798 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.422+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651c5c01e30 con 0x5651c5bf5b40 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.422+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa40d5540 con 0x5651c5bf5b40 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fefa40c8620 con 0x5651c5bf5b40 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 10 client.25798 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 10 client.25798 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 20 client.25798 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c10ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 4 client.25798 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fefa40e5070 con 0x5651c5c0eab0 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.423+0000 7fefae59c640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c10ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.424+0000 7fefae59c640 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c10ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77a1672cac759cf7 server_cookie=8fc81382f1d40be5 in_seq=0 out_seq=0 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.424+0000 7fef95ffb640 10 client.25798 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.425+0000 7fef95ffb640 1 -- 172.21.15.3:0/3350017726 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fefa40e5070 con 0x5651c5c0eab0 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.425+0000 7fef95ffb640 10 client.25798 handle_command_reply: tid=0 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.425+0000 7fefb7052740 1 client.25798 shutdown 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.426+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 msgr2=0x5651c5bfbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.426+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005ec0 tx=0x7fefa401a570 comp rx=0 tx=0).stop 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 shutdown_connections 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5bfc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf5b40 0x5651c5bfbfc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c10ea0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fef95ffb640 0 client.25798 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:21:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 --2- 172.21.15.3:0/3350017726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf45d0 0x5651c5bfed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 >> 172.21.15.3:0/3350017726 conn(0x5651c5bbd310 msgr2=0x5651c5bf69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 shutdown_connections 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.427+0000 7fefb7052740 1 -- 172.21.15.3:0/3350017726 wait complete. 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.428+0000 7fefb7052740 20 client.25798 trim_cache size 0 max 16384 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.452+0000 7fefb7052740 1 Processor -- start 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.452+0000 7fefb7052740 1 -- start start 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 0x5651c5c16700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c16c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef9000e540 con 0x5651c5bff370 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef90010170 con 0x5651c5bf5b40 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.453+0000 7fefb7052740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef9000ec10 con 0x5651c5bf45d0 2024-09-12T02:21:35.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 0x5651c5c16700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 0x5651c5c16700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38412/0 (socket says 172.21.15.3:38412) 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefadd9b640 1 -- 172.21.15.3:0/3940943629 learned_addr learned my addr 172.21.15.3:0/3940943629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefaed9d640 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 -- 172.21.15.3:0/3940943629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 msgr2=0x5651c5c16700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 0x5651c5c16700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 -- 172.21.15.3:0/3940943629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 msgr2=0x5651c5c16c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c16c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.454+0000 7fefae59c640 1 -- 172.21.15.3:0/3940943629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c5bfd340 con 0x5651c5bf45d0 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.455+0000 7fefaed9d640 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c16c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.456+0000 7fefae59c640 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c0b0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefa8008c70 tx=0x7fefa8011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2790e09892faf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.456+0000 7fef97fff640 1 -- 172.21.15.3:0/3940943629 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8021070 con 0x5651c5bf45d0 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.456+0000 7fef97fff640 1 -- 172.21.15.3:0/3940943629 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa8004d20 con 0x5651c5bf45d0 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.456+0000 7fef97fff640 1 -- 172.21.15.3:0/3940943629 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8005550 con 0x5651c5bf45d0 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.457+0000 7fefb7052740 1 -- 172.21.15.3:0/3940943629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 msgr2=0x5651c5c0b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.457+0000 7fefb7052740 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c0b0e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fefa8008c70 tx=0x7fefa8011a00 comp rx=0 tx=0).stop 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 -- 172.21.15.3:0/3940943629 shutdown_connections 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bff370 0x5651c5c16c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bf5b40 0x5651c5c16700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 --2- 172.21.15.3:0/3940943629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c0b0e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 -- 172.21.15.3:0/3940943629 >> 172.21.15.3:0/3940943629 conn(0x5651c5bbd310 msgr2=0x5651c5bfe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 -- 172.21.15.3:0/3940943629 shutdown_connections 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.458+0000 7fefb7052740 1 -- 172.21.15.3:0/3940943629 wait complete. 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.459+0000 7fefb7052740 1 -- start start 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.459+0000 7fefb7052740 1 Processor -- start 2024-09-12T02:21:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bf5b40 0x5651c5c07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fef9000fa10 con 0x5651c5bf5b40 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fef9000f340 con 0x5651c5bff370 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefaed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38420/0 (socket says 172.21.15.3:38420) 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefaed9d640 1 -- 172.21.15.3:0/593118935 learned_addr learned my addr 172.21.15.3:0/593118935 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fef9000c520 con 0x5651c5bf45d0 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefb7052740 10 client.0 _ll_register_callbacks cb 0x7fefa80085f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.460+0000 7fefadd9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bf5b40 0x5651c5c07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefae59c640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefaed9d640 1 -- 172.21.15.3:0/593118935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 msgr2=0x5651c5c07250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefaed9d640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c07250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefaed9d640 1 -- 172.21.15.3:0/593118935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bf5b40 msgr2=0x5651c5c07790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefaed9d640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bf5b40 0x5651c5c07790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.461+0000 7fefaed9d640 1 -- 172.21.15.3:0/593118935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c5c07dd0 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.462+0000 7fefaed9d640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefa0010b10 tx=0x7fefa0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3008c1245d95dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fef95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0016070 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fefa001cab0 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa8004d20 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fefa001ccf0 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fefb7052740 10 client.25664 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.463+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651c5c79c30 con 0x5651c5bff370 2024-09-12T02:21:35.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa002c070 con 0x5651c5bff370 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fefa0027430 con 0x5651c5bff370 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 10 client.25664 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 10 client.25664 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 20 client.25664 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 4 client.25664 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.465+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 --> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fefa8004d20 con 0x5651c5c0eab0 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.466+0000 7fefae59c640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.466+0000 7fefae59c640 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c7c420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=808f600b8799f53d server_cookie=ce7b2e64b85b75f in_seq=0 out_seq=0 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.466+0000 7fef95ffb640 10 client.25664 ms_handle_connect on v2:172.21.15.26:6838/1997151172 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fef95ffb640 1 -- 172.21.15.3:0/593118935 <== mds.3 v2:172.21.15.26:6838/1997151172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fefa8004d20 con 0x5651c5c0eab0 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fef95ffb640 10 client.25664 handle_command_reply: tid=0 2024-09-12T02:21:35.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 client.25664 shutdown 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 msgr2=0x5651c5c04220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefa0010b10 tx=0x7fefa0004730 comp rx=0 tx=0).stop 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 shutdown_connections 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:6838/1997151172,v1:172.21.15.26:6839/1997151172] conn(0x5651c5c0eab0 0x5651c5c7c420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c5bff370 0x5651c5c04220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5651c5bf5b40 0x5651c5c07790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 --2- 172.21.15.3:0/593118935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5651c5bf45d0 0x5651c5c07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 >> 172.21.15.3:0/593118935 conn(0x5651c5bbd310 msgr2=0x5651c5c0a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.680 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.467+0000 7fef95ffb640 0 client.25664 ms_handle_reset on v2:172.21.15.26:6838/1997151172 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.469+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 shutdown_connections 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.469+0000 7fefb7052740 1 -- 172.21.15.3:0/593118935 wait complete. 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.469+0000 7fefb7052740 20 client.25664 trim_cache size 0 max 16384 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 msgr2=0x7fef980a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 0x7fef980a5210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fefa0005fd0 tx=0x7fefa000a250 comp rx=0 tx=0).stop 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 msgr2=0x7fefb01167c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefa403b950 tx=0x7fefa4040d00 comp rx=0 tx=0).stop 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 shutdown_connections 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fef980a2d50 0x7fef980a5210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fefb011b140 0x7fefb0116d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb0109830 0x7fefb01167c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 --2- 172.21.15.3:0/2684015678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefb0109450 0x7fefb011a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.471+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 >> 172.21.15.3:0/2684015678 conn(0x7fefb006e140 msgr2=0x7fefb0070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.472+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 shutdown_connections 2024-09-12T02:21:35.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:34.472+0000 7fefb4c79640 1 -- 172.21.15.3:0/2684015678 wait complete. 2024-09-12T02:21:36.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:35 smithi026 ceph-mon[35496]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 5.26k op/s 2024-09-12T02:21:36.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:35 smithi003 ceph-mon[31476]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 5.26k op/s 2024-09-12T02:21:36.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:35 smithi092 ceph-mon[36089]: pgmap v432: 97 pgs: 97 active+clean; 13 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 5.26k op/s 2024-09-12T02:21:38.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:37 smithi026 ceph-mon[35496]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 6.68k op/s 2024-09-12T02:21:38.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:37 smithi003 ceph-mon[31476]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 6.68k op/s 2024-09-12T02:21:38.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:37 smithi092 ceph-mon[36089]: pgmap v433: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 6.68k op/s 2024-09-12T02:21:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.454+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/33550553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8102ba0 msgr2=0x7f7cd8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.454+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/33550553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8103020 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009900 tx=0x7f7cc002a410 comp rx=0 tx=0).stop 2024-09-12T02:21:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/33550553 shutdown_connections 2024-09-12T02:21:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/33550553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd8103560 0x7f7cd8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/33550553 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8103020 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/33550553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd81019b0 0x7f7cd8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/33550553 >> 172.21.15.3:0/33550553 conn(0x7f7cd80fd130 msgr2=0x7f7cd80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/33550553 shutdown_connections 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/33550553 wait complete. 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 Processor -- start 2024-09-12T02:21:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.455+0000 7f7cdcb9d640 1 -- start start 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 0x7f7cd8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810d630 con 0x7f7cd8103560 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810d4b0 con 0x7f7cd81019b0 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cdcb9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810d7b0 con 0x7f7cd8102ba0 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53626/0 (socket says 172.21.15.3:53626) 2024-09-12T02:21:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 -- 172.21.15.3:0/1832519357 learned_addr learned my addr 172.21.15.3:0/1832519357 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 0x7f7cd8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 -- 172.21.15.3:0/1832519357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 msgr2=0x7f7cd8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 -- 172.21.15.3:0/1832519357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 msgr2=0x7f7cd8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 0x7f7cd8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 -- 172.21.15.3:0/1832519357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd819f900 con 0x7f7cd8103560 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 0x7f7cd8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.456+0000 7f7cd7fff640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000eb10 tx=0x7f7cd000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.457+0000 7f7cd57fa640 1 -- 172.21.15.3:0/1832519357 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0004b20 con 0x7f7cd8103560 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.457+0000 7f7cd57fa640 1 -- 172.21.15.3:0/1832519357 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd0004cc0 con 0x7f7cd8103560 2024-09-12T02:21:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.457+0000 7f7cd57fa640 1 -- 172.21.15.3:0/1832519357 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0010760 con 0x7f7cd8103560 2024-09-12T02:21:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.457+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd819fbf0 con 0x7f7cd8103560 2024-09-12T02:21:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.457+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd81a00c0 con 0x7f7cd8103560 2024-09-12T02:21:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.459+0000 7f7cd57fa640 1 -- 172.21.15.3:0/1832519357 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7cd0005020 con 0x7f7cd8103560 2024-09-12T02:21:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.459+0000 7f7ce0c6f740 1 Processor -- start 2024-09-12T02:21:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.459+0000 7f7ce0c6f740 1 -- start start 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd5ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56434fcc32f0 con 0x56434fd65540 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56434fc6b270 con 0x56434fd604b0 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56434fcbf2b0 con 0x56434fd5fb70 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd67a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53638/0 (socket says 172.21.15.3:53638) 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cceffd640 1 -- 172.21.15.3:0/1729019391 learned_addr learned my addr 172.21.15.3:0/1729019391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:47504/0 (socket says 172.21.15.3:47504) 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd57fa640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 0x7f7cc40a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd5ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd7fff640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 0x7f7cc40a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 -- 172.21.15.3:0/1729019391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 msgr2=0x56434fd5ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd5ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 -- 172.21.15.3:0/1729019391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 msgr2=0x56434fd67a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd67a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd77fe640 1 -- 172.21.15.3:0/1729019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434fc6a340 con 0x56434fd604b0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.460+0000 7f7cd57fa640 1 -- 172.21.15.3:0/1832519357 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7cd00cf710 con 0x7f7cd8103560 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8102a00 tx=0x7f7cc80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af5e5e81fbbd29c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7cd7fff640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 0x7f7cc40a5030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00969a0 tx=0x7f7cd00c6a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7ccdffb640 1 -- 172.21.15.3:0/1729019391 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8004230 con 0x56434fd604b0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7ccdffb640 1 -- 172.21.15.3:0/1729019391 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc80043d0 con 0x56434fd604b0 2024-09-12T02:21:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7ccdffb640 1 -- 172.21.15.3:0/1729019391 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc800fb20 con 0x56434fd604b0 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1729019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 msgr2=0x56434fd64f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.461+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8102a00 tx=0x7f7cc80118d0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1729019391 shutdown_connections 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd67a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd64f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1729019391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd5ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1729019391 >> 172.21.15.3:0/1729019391 conn(0x56434fd5b1e0 msgr2=0x56434fd5b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1729019391 shutdown_connections 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1729019391 wait complete. 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 -- start start 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.462+0000 7f7ce0c6f740 1 Processor -- start 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd6a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd65540 0x56434fd71310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56434fd68510 con 0x56434fd5fb70 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56434fc4c290 con 0x56434fd604b0 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56434fd68690 con 0x56434fd65540 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 10 client.0 _ll_register_callbacks cb 0x56434fc7d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53648/0 (socket says 172.21.15.3:53648) 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 -- 172.21.15.3:0/1613187933 learned_addr learned my addr 172.21.15.3:0/1613187933 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd6a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd65540 0x56434fd71310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 -- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd65540 msgr2=0x56434fd71310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd65540 0x56434fd71310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 -- 172.21.15.3:0/1613187933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 msgr2=0x56434fd6a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd6a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 -- 172.21.15.3:0/1613187933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434f9d6210 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd6a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00314e0 tx=0x7f7cc002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce47d79a644d447 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0035040 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7cc00042d0 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.463+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc80043d0 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7cc0052050 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 10 client.16218 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56434faa2620 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc80043d0 con 0x56434fd5fb70 2024-09-12T02:21:38.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f7cc003a040 con 0x56434fd5fb70 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 10 client.16218 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 20 client.16218 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fd75a20 0x56434fd77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 4 client.16218 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.464+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56434fcd0a20 con 0x56434fd75a20 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.465+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fd75a20 0x56434fd77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.465+0000 7f7cb6ffd640 10 client.16218 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.465+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fd75a20 0x56434fd77e60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3491d94254568df2 server_cookie=f87596dffe360dbe in_seq=0 out_seq=0 2024-09-12T02:21:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1613187933 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56434fcd0a20 con 0x56434fd75a20 2024-09-12T02:21:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7cb6ffd640 10 client.16218 handle_command_reply: tid=0 2024-09-12T02:21:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 client.16218 shutdown 2024-09-12T02:21:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 msgr2=0x56434fd69d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00314e0 tx=0x7f7cc002a410 comp rx=0 tx=0).stop 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 shutdown_connections 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fd75a20 0x56434fd77e60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd65540 0x56434fd71310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd6a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7cb6ffd640 0 client.16218 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1613187933 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd5fb70 0x56434fd69d90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 >> 172.21.15.3:0/1613187933 conn(0x56434fd5b1e0 msgr2=0x56434fd5c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 shutdown_connections 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1613187933 wait complete. 2024-09-12T02:21:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.466+0000 7f7ce0c6f740 20 client.16218 trim_cache size 0 max 16384 2024-09-12T02:21:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 Processor -- start 2024-09-12T02:21:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 -- start start 2024-09-12T02:21:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd6a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd71230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81a8e80 con 0x56434fd65540 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81aaab0 con 0x56434fd604b0 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81a9550 con 0x56434fd5fb70 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53662/0 (socket says 172.21.15.3:53662) 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 -- 172.21.15.3:0/2881864524 learned_addr learned my addr 172.21.15.3:0/2881864524 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd6a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd71230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 -- 172.21.15.3:0/2881864524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 msgr2=0x56434fd6a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd6a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 -- 172.21.15.3:0/2881864524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 msgr2=0x56434fd71230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd71230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cceffd640 1 -- 172.21.15.3:0/2881864524 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56434fd698d0 con 0x56434fd65540 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.481+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7cceffd640 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005ec0 tx=0x7f7cc0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d933b82c403571a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ccd7fa640 1 -- 172.21.15.3:0/2881864524 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0035030 con 0x56434fd65540 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ccd7fa640 1 -- 172.21.15.3:0/2881864524 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc003d8e0 con 0x56434fd65540 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ccd7fa640 1 -- 172.21.15.3:0/2881864524 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0031bb0 con 0x56434fd65540 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/2881864524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 msgr2=0x56434fd59a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0005ec0 tx=0x7f7cc0031400 comp rx=0 tx=0).stop 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/2881864524 shutdown_connections 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd59a90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd604b0 0x56434fd71230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/2881864524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd5fb70 0x56434fd6a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/2881864524 >> 172.21.15.3:0/2881864524 conn(0x56434fd5b1e0 msgr2=0x56434fd69b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/2881864524 shutdown_connections 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.482+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/2881864524 wait complete. 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 -- start start 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 Processor -- start 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd5fb70 0x56434fd6e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd604b0 0x56434fd6ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81aa350 con 0x56434fd65540 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81a9c80 con 0x56434fd5fb70 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd81a6e60 con 0x56434fd604b0 2024-09-12T02:21:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7ce0c6f740 10 client.0 _ll_register_callbacks cb 0x7f7cc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7cceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53676/0 (socket says 172.21.15.3:53676) 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7cceffd640 1 -- 172.21.15.3:0/1911730342 learned_addr learned my addr 172.21.15.3:0/1911730342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd5fb70 0x56434fd6e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.483+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd604b0 0x56434fd6ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 -- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd604b0 msgr2=0x56434fd6ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd604b0 0x56434fd6ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 -- 172.21.15.3:0/1911730342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd5fb70 msgr2=0x56434fd6e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd5fb70 0x56434fd6e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 -- 172.21.15.3:0/1911730342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56434fd70250 con 0x56434fd65540 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cceffd640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0031920 tx=0x7f7cc00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e2339855ca5904 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0035070 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f7cc0031db0 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc003d8e0 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7cc00038c0 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7ce0c6f740 10 client.16224 fetch_fsmap learned FSMap version 33 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56434fd714d0 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.484+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cc003d8e0 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f7cc0056a70 con 0x56434fd65540 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 10 client.16224 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 20 client.16224 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fde0ca0 0x56434fde30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 4 client.16224 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:21:38.490 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56434fccad70 con 0x56434fde0ca0 2024-09-12T02:21:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fde0ca0 0x56434fde30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7cb6ffd640 10 client.16224 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:38.491 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.485+0000 7f7cd77fe640 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fde0ca0 0x56434fde30c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0ceff106870138 server_cookie=c6da5351b3c22685 in_seq=0 out_seq=0 2024-09-12T02:21:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7cb6ffd640 1 -- 172.21.15.3:0/1911730342 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56434fccad70 con 0x56434fde0ca0 2024-09-12T02:21:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7cb6ffd640 10 client.16224 handle_command_reply: tid=0 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 client.16224 shutdown 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 msgr2=0x56434fd6fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0031920 tx=0x7f7cc00092c0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 shutdown_connections 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x56434fde0ca0 0x56434fde30c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56434fd65540 0x56434fd6fc40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56434fd604b0 0x56434fd6ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7cb6ffd640 0 client.16224 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 --2- 172.21.15.3:0/1911730342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56434fd5fb70 0x56434fd6e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 >> 172.21.15.3:0/1911730342 conn(0x56434fd5b1e0 msgr2=0x56434fd5c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 shutdown_connections 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 1 -- 172.21.15.3:0/1911730342 wait complete. 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.495+0000 7f7ce0c6f740 20 client.16224 trim_cache size 0 max 16384 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:21:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 msgr2=0x7f7cc40a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 0x7f7cc40a5030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00969a0 tx=0x7f7cd00c6a90 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 msgr2=0x7f7cd819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000eb10 tx=0x7f7cd000c2d0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 shutdown_connections 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f7cc40a2b70 0x7f7cc40a5030 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cd8103560 0x7f7cd819f260 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cd8102ba0 0x7f7cd8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 --2- 172.21.15.3:0/1832519357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd81019b0 0x7f7cd8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 >> 172.21.15.3:0/1832519357 conn(0x7f7cd80fd130 msgr2=0x7f7cd8105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 shutdown_connections 2024-09-12T02:21:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:38.496+0000 7f7cdcb9d640 1 -- 172.21.15.3:0/1832519357 wait complete. 2024-09-12T02:21:40.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:39 smithi026 ceph-mon[35496]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 37 MiB/s wr, 6.15k op/s 2024-09-12T02:21:40.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:39 smithi003 ceph-mon[31476]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 37 MiB/s wr, 6.15k op/s 2024-09-12T02:21:40.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:39 smithi092 ceph-mon[36089]: pgmap v434: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 37 MiB/s wr, 6.15k op/s 2024-09-12T02:21:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:41 smithi026 ceph-mon[35496]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-09-12T02:21:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:41 smithi003 ceph-mon[31476]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-09-12T02:21:42.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:41 smithi092 ceph-mon[36089]: pgmap v435: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 6.64k op/s 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 34 MiB/s wr, 5.97k op/s 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:21:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 34 MiB/s wr, 5.97k op/s 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:21:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: pgmap v436: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 34 MiB/s wr, 5.97k op/s 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:21:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:21:44.132 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db --set-id d6859252 --release --await' 2024-09-12T02:21:44.132 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db --set-id d6859252 --release --await 2024-09-12T02:21:46.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:45 smithi026 ceph-mon[35496]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 5.23k op/s 2024-09-12T02:21:46.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:45 smithi003 ceph-mon[31476]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 5.23k op/s 2024-09-12T02:21:46.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:45 smithi092 ceph-mon[36089]: pgmap v437: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 29 MiB/s wr, 5.23k op/s 2024-09-12T02:21:48.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:47 smithi026 ceph-mon[35496]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 5.77k op/s 2024-09-12T02:21:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:47 smithi003 ceph-mon[31476]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 5.77k op/s 2024-09-12T02:21:48.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:47 smithi092 ceph-mon[36089]: pgmap v438: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 5.77k op/s 2024-09-12T02:21:49.177 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:21:49.216 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:49.759 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:49 smithi003 ceph-mon[31476]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 MiB/s wr, 4.06k op/s 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 -- 172.21.15.3:0/2911192653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109e00 msgr2=0x7f189810ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 --2- 172.21.15.3:0/2911192653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109e00 0x7f189810ebe0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02a440 comp rx=0 tx=0).stop 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 -- 172.21.15.3:0/2911192653 shutdown_connections 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 --2- 172.21.15.3:0/2911192653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 --2- 172.21.15.3:0/2911192653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109e00 0x7f189810ebe0 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02a440 comp rx=0 tx=0).stop 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 --2- 172.21.15.3:0/2911192653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898109450 0x7f1898109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.846+0000 7f189deec640 1 -- 172.21.15.3:0/2911192653 >> 172.21.15.3:0/2911192653 conn(0x7f189806e2d0 msgr2=0x7f189806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.851+0000 7f189deec640 1 -- 172.21.15.3:0/2911192653 shutdown_connections 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.851+0000 7f189deec640 1 -- 172.21.15.3:0/2911192653 wait complete. 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.851+0000 7f189deec640 1 Processor -- start 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 -- start start 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189811eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 0x7f189811f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898124ac0 0x7f1898119ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1898118610 con 0x7f1898109450 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1898118490 con 0x7f1898124ac0 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.852+0000 7f189deec640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1898118790 con 0x7f1898069ab0 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.854+0000 7f18977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189811eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.854+0000 7f1896ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 0x7f189811f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.854+0000 7f1896ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 0x7f189811f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39020/0 (socket says 172.21.15.3:39020) 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.854+0000 7f1896ffd640 1 -- 172.21.15.3:0/972754603 learned_addr learned my addr 172.21.15.3:0/972754603 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.855+0000 7f1897fff640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898124ac0 0x7f1898119ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 -- 172.21.15.3:0/972754603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898124ac0 msgr2=0x7f1898119ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898124ac0 0x7f1898119ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 -- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 msgr2=0x7f189811f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 0x7f189811f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 -- 172.21.15.3:0/972754603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189811a400 con 0x7f1898069ab0 2024-09-12T02:21:49.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.856+0000 7f18977fe640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189811eda0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1888009fd0 tx=0x7f188800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.859+0000 7f1894ff9640 1 -- 172.21.15.3:0/972754603 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888015020 con 0x7f1898069ab0 2024-09-12T02:21:49.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.859+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189811a6f0 con 0x7f1898069ab0 2024-09-12T02:21:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.859+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189811ac00 con 0x7f1898069ab0 2024-09-12T02:21:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.859+0000 7f1894ff9640 1 -- 172.21.15.3:0/972754603 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888011d10 con 0x7f1898069ab0 2024-09-12T02:21:49.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.859+0000 7f1894ff9640 1 -- 172.21.15.3:0/972754603 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188800f2e0 con 0x7f1898069ab0 2024-09-12T02:21:49.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.868+0000 7f18a02c5740 1 Processor -- start 2024-09-12T02:21:49.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.869+0000 7f1894ff9640 1 -- 172.21.15.3:0/972754603 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1888018020 con 0x7f1898069ab0 2024-09-12T02:21:49.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.870+0000 7f18a02c5740 1 -- start start 2024-09-12T02:21:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.873+0000 7f1894ff9640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 0x7f18800a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.873+0000 7f1894ff9640 1 -- 172.21.15.3:0/972754603 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f18880ce700 con 0x7f1898069ab0 2024-09-12T02:21:49.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.873+0000 7f1897fff640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 0x7f18800a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22c5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f5f222a620 con 0x55f5f22c5060 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5f21d1e50 con 0x55f5f22c5980 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f18a02c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5f22265e0 con 0x55f5f22d0240 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56778/0 (socket says 172.21.15.3:56778) 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 -- 172.21.15.3:0/857328829 learned_addr learned my addr 172.21.15.3:0/857328829 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:49.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 0x7f18800a51d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f189000ae10 tx=0x7f189000a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 -- 172.21.15.3:0/857328829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 msgr2=0x55f5f22cfd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.874+0000 7f1897fff640 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cfd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.875+0000 7f1897fff640 1 -- 172.21.15.3:0/857328829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 msgr2=0x55f5f22c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.875+0000 7f1897fff640 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.875+0000 7f1897fff640 1 -- 172.21.15.3:0/857328829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5f21c2d40 con 0x55f5f22d0240 2024-09-12T02:21:49.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.875+0000 7f1897fff640 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1890010680 tx=0x7f1890012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3c8866af4e9cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.877+0000 7f187dffb640 1 -- 172.21.15.3:0/857328829 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890016070 con 0x55f5f22d0240 2024-09-12T02:21:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.877+0000 7f187dffb640 1 -- 172.21.15.3:0/857328829 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189001bd20 con 0x55f5f22d0240 2024-09-12T02:21:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.877+0000 7f187dffb640 1 -- 172.21.15.3:0/857328829 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890004720 con 0x55f5f22d0240 2024-09-12T02:21:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.879+0000 7f18a02c5740 1 -- 172.21.15.3:0/857328829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 msgr2=0x55f5f22d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.879+0000 7f18a02c5740 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1890010680 tx=0x7f1890012000 comp rx=0 tx=0).stop 2024-09-12T02:21:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.882+0000 7f18a02c5740 1 -- 172.21.15.3:0/857328829 shutdown_connections 2024-09-12T02:21:49.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.882+0000 7f18a02c5740 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22d2630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.882+0000 7f18a02c5740 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.882+0000 7f18a02c5740 1 --2- 172.21.15.3:0/857328829 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.882+0000 7f18a02c5740 1 -- 172.21.15.3:0/857328829 >> 172.21.15.3:0/857328829 conn(0x55f5f22c15f0 msgr2=0x55f5f22c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:49.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.883+0000 7f18a02c5740 1 -- 172.21.15.3:0/857328829 shutdown_connections 2024-09-12T02:21:49.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.883+0000 7f18a02c5740 1 -- 172.21.15.3:0/857328829 wait complete. 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 -- start start 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 Processor -- start 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f5f22d2e10 con 0x55f5f22c5060 2024-09-12T02:21:49.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f5f21b2f10 con 0x55f5f22c5980 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5f22d2f90 con 0x55f5f22d0240 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.884+0000 7f18a02c5740 10 client.0 _ll_register_callbacks cb 0x55f5f21d6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39042/0 (socket says 172.21.15.3:39042) 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 -- 172.21.15.3:0/971666897 learned_addr learned my addr 172.21.15.3:0/971666897 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f1897fff640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f1896ffd640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 -- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 msgr2=0x55f5f22cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 -- 172.21.15.3:0/971666897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 msgr2=0x55f5f22cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.885+0000 7f18977fe640 1 -- 172.21.15.3:0/971666897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5f1f39580 con 0x55f5f22c5060 2024-09-12T02:21:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.886+0000 7f18977fe640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f18880a3690 tx=0x7f188809a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefba01c97ae428a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.886+0000 7f1876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.887+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188809f300 con 0x55f5f22c5060 2024-09-12T02:21:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.887+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1888099250 con 0x55f5f22c5060 2024-09-12T02:21:49.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.887+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189001bd20 con 0x55f5f22c5060 2024-09-12T02:21:49.890 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:21:49.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.890+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f18880a3bc0 con 0x55f5f22c5060 2024-09-12T02:21:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.895+0000 7f18a02c5740 10 client.16233 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:49.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.895+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f5f2003960 con 0x55f5f22c5060 2024-09-12T02:21:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189001bd20 con 0x55f5f22c5060 2024-09-12T02:21:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f188809a3a0 con 0x55f5f22c5060 2024-09-12T02:21:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 10 client.16233 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 10 client.16233 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:21:49.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 20 client.16233 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f22dfea0 0x55f5f22e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 4 client.16233 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:49.899 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.896+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5f2237b90 con 0x55f5f22dfea0 2024-09-12T02:21:49.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.898+0000 7f1896ffd640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f22dfea0 0x55f5f22e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.902+0000 7f1896ffd640 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f22dfea0 0x55f5f22e22b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef964a2c2969bc9c server_cookie=5825597b97977136 in_seq=0 out_seq=0 2024-09-12T02:21:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.902+0000 7f1876ffd640 10 client.16233 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:49.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.903+0000 7f1876ffd640 1 -- 172.21.15.3:0/971666897 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5f2237b90 con 0x55f5f22dfea0 2024-09-12T02:21:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.903+0000 7f1876ffd640 10 client.16233 handle_command_reply: tid=0 2024-09-12T02:21:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.903+0000 7f18a02c5740 1 client.16233 shutdown 2024-09-12T02:21:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.904+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 msgr2=0x55f5f22cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.905 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.904+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f18880a3690 tx=0x7f188809a970 comp rx=0 tx=0).stop 2024-09-12T02:21:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 shutdown_connections 2024-09-12T02:21:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f22dfea0 0x55f5f22e22b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22d0240 0x55f5f22cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 --2- 172.21.15.3:0/971666897 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5060 0x55f5f22cd160 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 >> 172.21.15.3:0/971666897 conn(0x55f5f22c15f0 msgr2=0x55f5f22c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f1876ffd640 0 client.16233 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 shutdown_connections 2024-09-12T02:21:49.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 1 -- 172.21.15.3:0/971666897 wait complete. 2024-09-12T02:21:49.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.910+0000 7f18a02c5740 20 client.16233 trim_cache size 0 max 16384 2024-09-12T02:21:49.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.948+0000 7f18a02c5740 1 Processor -- start 2024-09-12T02:21:49.950 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 -- start start 2024-09-12T02:21:49.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5980 0x55f5f22cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22d0240 0x55f5f22e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f188400e590 con 0x55f5f22c5980 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1884010190 con 0x55f5f22d0240 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18a02c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f188400ec60 con 0x55f5f22c5060 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56790/0 (socket says 172.21.15.3:56790) 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f18977fe640 1 -- 172.21.15.3:0/2090543697 learned_addr learned my addr 172.21.15.3:0/2090543697 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.949+0000 7f1897fff640 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22d0240 0x55f5f22e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.950+0000 7f18977fe640 1 -- 172.21.15.3:0/2090543697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22d0240 msgr2=0x55f5f22e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.950+0000 7f18977fe640 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22d0240 0x55f5f22e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.950+0000 7f18977fe640 1 -- 172.21.15.3:0/2090543697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5980 msgr2=0x55f5f22cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.950+0000 7f18977fe640 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5980 0x55f5f22cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.950+0000 7f18977fe640 1 -- 172.21.15.3:0/2090543697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5f22c71e0 con 0x55f5f22c5060 2024-09-12T02:21:49.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.954+0000 7f18977fe640 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f188800bd70 tx=0x7f188809a9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1c4fca3503bc75d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.956+0000 7f187d7fa640 1 -- 172.21.15.3:0/2090543697 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888099250 con 0x55f5f22c5060 2024-09-12T02:21:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.956+0000 7f187d7fa640 1 -- 172.21.15.3:0/2090543697 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18880975c0 con 0x55f5f22c5060 2024-09-12T02:21:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.957+0000 7f18a02c5740 1 -- 172.21.15.3:0/2090543697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 msgr2=0x55f5f22cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:49.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.957+0000 7f18a02c5740 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f188800bd70 tx=0x7f188809a9e0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.960+0000 7f18a02c5740 1 -- 172.21.15.3:0/2090543697 shutdown_connections 2024-09-12T02:21:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.960+0000 7f18a02c5740 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22d0240 0x55f5f22e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.960+0000 7f18a02c5740 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22c5980 0x55f5f22cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.960+0000 7f18a02c5740 1 --2- 172.21.15.3:0/2090543697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22cba30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.960+0000 7f18a02c5740 1 -- 172.21.15.3:0/2090543697 >> 172.21.15.3:0/2090543697 conn(0x55f5f22c15f0 msgr2=0x55f5f22ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.964+0000 7f18a02c5740 1 -- 172.21.15.3:0/2090543697 shutdown_connections 2024-09-12T02:21:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.964+0000 7f18a02c5740 1 -- 172.21.15.3:0/2090543697 wait complete. 2024-09-12T02:21:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.964+0000 7f18a02c5740 1 -- start start 2024-09-12T02:21:49.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 Processor -- start 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f188400fa30 con 0x55f5f22d0240 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f188400f360 con 0x55f5f22c5980 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.965+0000 7f18a02c5740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f188400c570 con 0x55f5f22c5060 2024-09-12T02:21:49.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.966+0000 7f18a02c5740 10 client.0 _ll_register_callbacks cb 0x7f188809aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:49.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.966+0000 7f18977fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.966+0000 7f1897fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.966+0000 7f1897fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39060/0 (socket says 172.21.15.3:39060) 2024-09-12T02:21:49.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.966+0000 7f1897fff640 1 -- 172.21.15.3:0/383170572 learned_addr learned my addr 172.21.15.3:0/383170572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:49.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.967+0000 7f1896ffd640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.970+0000 7f1897fff640 1 -- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 msgr2=0x55f5f22d85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.970+0000 7f1897fff640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22d85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.970+0000 7f1897fff640 1 -- 172.21.15.3:0/383170572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 msgr2=0x55f5f22d8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.970+0000 7f1897fff640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22d8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.970+0000 7f1897fff640 1 -- 172.21.15.3:0/383170572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5f22cc110 con 0x55f5f22d0240 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.971+0000 7f18977fe640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:21:49.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.971+0000 7f1896ffd640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.974+0000 7f1897fff640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1890010680 tx=0x7f189001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e074c7a6a791ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.974+0000 7f1876ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.976+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890016040 con 0x55f5f22d0240 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.976+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18880975c0 con 0x55f5f22d0240 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.976+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f189001bb60 con 0x55f5f22d0240 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.979+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1890003970 con 0x55f5f22d0240 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.980+0000 7f18a02c5740 10 client.16236 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:49.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.980+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f5f22c57b0 con 0x55f5f22d0240 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18880975c0 con 0x55f5f22d0240 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1890004660 con 0x55f5f22d0240 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 10 client.16236 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 10 client.16236 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 20 client.16236 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f234bbd0 0x55f5f234dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:49.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 4 client.16236 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d6859252", "release": true, "await": true}] 2024-09-12T02:21:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.981+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d6859252", "release": true, "await": true}) -- 0x55f5f22320a0 con 0x55f5f234bbd0 2024-09-12T02:21:49.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.983+0000 7f18977fe640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f234bbd0 0x55f5f234dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:49.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.988+0000 7f1876ffd640 10 client.16236 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:49.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:49.988+0000 7f18977fe640 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f234bbd0 0x55f5f234dff0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9769eaee77d85b4c server_cookie=9b1a2ce8ba8ea0fe in_seq=0 out_seq=0 2024-09-12T02:21:50.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:49 smithi026 ceph-mon[35496]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 MiB/s wr, 4.06k op/s 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.018+0000 7f1876ffd640 1 -- 172.21.15.3:0/383170572 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f5f22320a0 con 0x55f5f234bbd0 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.018+0000 7f1876ffd640 10 client.16236 handle_command_reply: tid=0 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.018+0000 7f18a02c5740 1 client.16236 shutdown 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 msgr2=0x55f5f22d5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1890010680 tx=0x7f189001ba70 comp rx=0 tx=0).stop 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 shutdown_connections 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f5f234bbd0 0x55f5f234dff0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f5f22d0240 0x55f5f22d5410 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f1876ffd640 0 client.16236 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5f22c5980 0x55f5f22d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 --2- 172.21.15.3:0/383170572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5f22c5060 0x55f5f22d85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 >> 172.21.15.3:0/383170572 conn(0x55f5f22c15f0 msgr2=0x55f5f22cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 shutdown_connections 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 1 -- 172.21.15.3:0/383170572 wait complete. 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.019+0000 7f18a02c5740 20 client.16236 trim_cache size 0 max 16384 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:21:50.021 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 34, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 8, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "d6859252": { 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 8, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 133.4, 2024-09-12T02:21:50.022 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.023 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 msgr2=0x7f18800a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 0x7f18800a51d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f189000ae10 tx=0x7f189000a4d0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 msgr2=0x7f189811eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189811eda0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1888009fd0 tx=0x7f188800b870 comp rx=0 tx=0).stop 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 shutdown_connections 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18800a2d10 0x7f18800a51d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1898124ac0 0x7f1898119ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1898109450 0x7f189811f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 --2- 172.21.15.3:0/972754603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1898069ab0 0x7f189811eda0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.020+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 >> 172.21.15.3:0/972754603 conn(0x7f189806e2d0 msgr2=0x7f189810c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.021+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 shutdown_connections 2024-09-12T02:21:50.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.021+0000 7f189deec640 1 -- 172.21.15.3:0/972754603 wait complete. 2024-09-12T02:21:50.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:49 smithi092 ceph-mon[36089]: pgmap v439: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 22 MiB/s wr, 4.06k op/s 2024-09-12T02:21:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.771+0000 7f18a5ec4640 1 -- 172.21.15.3:0/3862348065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a0109450 msgr2=0x7f18a0110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.771+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/3862348065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a0109450 0x7f18a0110250 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402a420 comp rx=0 tx=0).stop 2024-09-12T02:21:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.772+0000 7f18a5ec4640 1 -- 172.21.15.3:0/3862348065 shutdown_connections 2024-09-12T02:21:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.772+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/3862348065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a011b140 0x7f18a011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.772+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/3862348065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.772+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/3862348065 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a0109450 0x7f18a0110250 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.772+0000 7f18a5ec4640 1 -- 172.21.15.3:0/3862348065 >> 172.21.15.3:0/3862348065 conn(0x7f18a006e140 msgr2=0x7f18a006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.776+0000 7f18a5ec4640 1 -- 172.21.15.3:0/3862348065 shutdown_connections 2024-09-12T02:21:50.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.777+0000 7f18a5ec4640 1 -- 172.21.15.3:0/3862348065 wait complete. 2024-09-12T02:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.778+0000 7f18a5ec4640 1 Processor -- start 2024-09-12T02:21:50.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.779+0000 7f18a5ec4640 1 -- start start 2024-09-12T02:21:50.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.779+0000 7f18a5ec4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a0109450 0x7f18a01a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.779+0000 7f18a5ec4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.779+0000 7f18a5ec4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 0x7f18a01abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f189f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a0109450 0x7f18a01a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f189effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f189effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56820/0 (socket says 172.21.15.3:56820) 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f189effd640 1 -- 172.21.15.3:0/2230903855 learned_addr learned my addr 172.21.15.3:0/2230903855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0120e30 con 0x7f18a011b140 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0120cb0 con 0x7f18a0109450 2024-09-12T02:21:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.780+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0120fb0 con 0x7f18a0109830 2024-09-12T02:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 0x7f18a01abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 -- 172.21.15.3:0/2230903855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 msgr2=0x7f18a01a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 -- 172.21.15.3:0/2230903855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a0109450 msgr2=0x7f18a01a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a0109450 0x7f18a01a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.781+0000 7f189ffff640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01ac300 con 0x7f18a011b140 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.782+0000 7f189effd640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:21:50.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.782+0000 7f189ffff640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 0x7f18a01abc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1898009fd0 tx=0x7f189800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.783+0000 7f189cff9640 1 -- 172.21.15.3:0/2230903855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898004d20 con 0x7f18a011b140 2024-09-12T02:21:50.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.783+0000 7f189cff9640 1 -- 172.21.15.3:0/2230903855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1898005020 con 0x7f18a011b140 2024-09-12T02:21:50.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.784+0000 7f189cff9640 1 -- 172.21.15.3:0/2230903855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980116d0 con 0x7f18a011b140 2024-09-12T02:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.784+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a01ac5f0 con 0x7f18a011b140 2024-09-12T02:21:50.788 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.784+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01b3ed0 con 0x7f18a011b140 2024-09-12T02:21:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.788+0000 7f18a829d740 1 Processor -- start 2024-09-12T02:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.792+0000 7f189cff9640 1 -- 172.21.15.3:0/2230903855 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1898005540 con 0x7f18a011b140 2024-09-12T02:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.793+0000 7f189cff9640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 0x7f18880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f189cff9640 1 -- 172.21.15.3:0/2230903855 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f18980cf3b0 con 0x7f18a011b140 2024-09-12T02:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 -- start start 2024-09-12T02:21:50.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d5eb732f0 con 0x560d5ec0eb40 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d5eb1acf0 con 0x560d5ec0d5d0 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f18a829d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d5eb6f2b0 con 0x560d5ec18370 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f189effd640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 0x7f18880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:39402/0 (socket says 172.21.15.3:39402) 2024-09-12T02:21:50.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.794+0000 7f189effd640 1 -- 172.21.15.3:0/1765874056 learned_addr learned my addr 172.21.15.3:0/1765874056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.797+0000 7f189effd640 1 -- 172.21.15.3:0/1765874056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 msgr2=0x560d5ec1a760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:21:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.797+0000 7f189effd640 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec1a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.797+0000 7f189effd640 1 -- 172.21.15.3:0/1765874056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 msgr2=0x560d5ec0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.797+0000 7f189effd640 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.797+0000 7f189effd640 1 -- 172.21.15.3:0/1765874056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d5eaff3a0 con 0x560d5ec0d5d0 2024-09-12T02:21:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.798+0000 7f189effd640 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1890016770 tx=0x7f1890019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3db667602dd69e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.798+0000 7f189effd640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 0x7f18880a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f18900146d0 tx=0x7f18900099f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.799+0000 7f1885ffb640 1 -- 172.21.15.3:0/1765874056 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18900187f0 con 0x560d5ec0d5d0 2024-09-12T02:21:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.799+0000 7f1885ffb640 1 -- 172.21.15.3:0/1765874056 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189001e070 con 0x560d5ec0d5d0 2024-09-12T02:21:50.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.799+0000 7f1885ffb640 1 -- 172.21.15.3:0/1765874056 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890022a70 con 0x560d5ec0d5d0 2024-09-12T02:21:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.800+0000 7f18a829d740 1 -- 172.21.15.3:0/1765874056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 msgr2=0x560d5ec0da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.800+0000 7f18a829d740 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1890016770 tx=0x7f1890019000 comp rx=0 tx=0).stop 2024-09-12T02:21:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.801+0000 7f18a829d740 1 -- 172.21.15.3:0/1765874056 shutdown_connections 2024-09-12T02:21:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.801+0000 7f18a829d740 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.801+0000 7f18a829d740 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0d5d0 0x560d5ec0da10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.801+0000 7f18a829d740 1 --2- 172.21.15.3:0/1765874056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.801+0000 7f18a829d740 1 -- 172.21.15.3:0/1765874056 >> 172.21.15.3:0/1765874056 conn(0x560d5ebd6310 msgr2=0x560d5ec09c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.805+0000 7f18a829d740 1 -- 172.21.15.3:0/1765874056 shutdown_connections 2024-09-12T02:21:50.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.805+0000 7f18a829d740 1 -- 172.21.15.3:0/1765874056 wait complete. 2024-09-12T02:21:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.806+0000 7f18a829d740 1 -- start start 2024-09-12T02:21:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.806+0000 7f18a829d740 1 Processor -- start 2024-09-12T02:21:50.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.806+0000 7f18a829d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec0d5d0 0x560d5ec17d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec18370 0x560d5ec15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d5ec1afb0 con 0x560d5ec0eb40 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d5eb2e060 con 0x560d5ec18370 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d5ec1b130 con 0x560d5ec0d5d0 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f18a829d740 10 client.0 _ll_register_callbacks cb 0x560d5e8818c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f189effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f189effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39108/0 (socket says 172.21.15.3:39108) 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f189effd640 1 -- 172.21.15.3:0/3759154877 learned_addr learned my addr 172.21.15.3:0/3759154877 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.807+0000 7f189ffff640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec18370 0x560d5ec15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 -- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec0d5d0 msgr2=0x560d5ec17d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec0d5d0 0x560d5ec17d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 -- 172.21.15.3:0/3759154877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec18370 msgr2=0x560d5ec15650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec18370 0x560d5ec15650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 -- 172.21.15.3:0/3759154877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d5e8e43c0 con 0x560d5ec0eb40 2024-09-12T02:21:50.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.808+0000 7f189effd640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1890024040 tx=0x7f1890016f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f224e3b8e1c791 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f187effd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890030020 con 0x560d5ec0eb40 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1890004040 con 0x560d5ec0eb40 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f189001e070 con 0x560d5ec0eb40 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f189003b050 con 0x560d5ec0eb40 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f18a829d740 10 client.16245 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.809+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560d5ec26a80 con 0x560d5ec0eb40 2024-09-12T02:21:50.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f189001e070 con 0x560d5ec0eb40 2024-09-12T02:21:50.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1890022cd0 con 0x560d5ec0eb40 2024-09-12T02:21:50.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 10 client.16245 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 20 client.16245 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec27e30 0x560d5ec2a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 4 client.16245 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.810+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d5eb80860 con 0x560d5ec27e30 2024-09-12T02:21:50.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.812+0000 7f189f7fe640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec27e30 0x560d5ec2a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.815+0000 7f189f7fe640 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec27e30 0x560d5ec2a220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c501735eb0780f62 server_cookie=7acb59b68db21874 in_seq=0 out_seq=0 2024-09-12T02:21:50.816 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.815+0000 7f187effd640 10 client.16245 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:21:50.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.819+0000 7f187effd640 1 -- 172.21.15.3:0/3759154877 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d5eb80860 con 0x560d5ec27e30 2024-09-12T02:21:50.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.819+0000 7f187effd640 10 client.16245 handle_command_reply: tid=0 2024-09-12T02:21:50.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.819+0000 7f18a829d740 1 client.16245 shutdown 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 msgr2=0x560d5ec15040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1890024040 tx=0x7f1890016f00 comp rx=0 tx=0).stop 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 shutdown_connections 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec18370 0x560d5ec15650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0eb40 0x560d5ec15040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec27e30 0x560d5ec2a220 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 --2- 172.21.15.3:0/3759154877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec0d5d0 0x560d5ec17d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 >> 172.21.15.3:0/3759154877 conn(0x560d5ebd6310 msgr2=0x560d5ec0f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.820+0000 7f187effd640 0 client.16245 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:21:50.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.821+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 shutdown_connections 2024-09-12T02:21:50.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.821+0000 7f18a829d740 1 -- 172.21.15.3:0/3759154877 wait complete. 2024-09-12T02:21:50.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.821+0000 7f18a829d740 20 client.16245 trim_cache size 0 max 16384 2024-09-12T02:21:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.846+0000 7f18a829d740 1 Processor -- start 2024-09-12T02:21:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.846+0000 7f18a829d740 1 -- start start 2024-09-12T02:21:50.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f18a829d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec14f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f18a829d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec24250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f18a829d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f18a829d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01bc240 con 0x560d5ec0d5d0 2024-09-12T02:21:50.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec24250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec24250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:39412/0 (socket says 172.21.15.3:39412) 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.847+0000 7f189effd640 1 -- 172.21.15.3:0/2790648531 learned_addr learned my addr 172.21.15.3:0/2790648531 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.848+0000 7f189ffff640 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.848+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01bde40 con 0x560d5ec0eb40 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.848+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01bc910 con 0x560d5ec18370 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.848+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec14f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 -- 172.21.15.3:0/2790648531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 msgr2=0x560d5ec10120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec10120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 -- 172.21.15.3:0/2790648531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 msgr2=0x560d5ec24250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec24250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 -- 172.21.15.3:0/2790648531 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d5ec13590 con 0x560d5ec0d5d0 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.849+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec14f70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1894009850 tx=0x7f1894000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dcbec4b16e5d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.850+0000 7f18857fa640 1 -- 172.21.15.3:0/2790648531 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035040 con 0x560d5ec0d5d0 2024-09-12T02:21:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.850+0000 7f18857fa640 1 -- 172.21.15.3:0/2790648531 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894048070 con 0x560d5ec0d5d0 2024-09-12T02:21:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.850+0000 7f18857fa640 1 -- 172.21.15.3:0/2790648531 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894004de0 con 0x560d5ec0d5d0 2024-09-12T02:21:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.851+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 msgr2=0x560d5ec14f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.851+0000 7f18a829d740 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec14f70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1894009850 tx=0x7f1894000bc0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.852+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 shutdown_connections 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.852+0000 7f18a829d740 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec10120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.852+0000 7f18a829d740 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec24250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.852+0000 7f18a829d740 1 --2- 172.21.15.3:0/2790648531 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec14f70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.852+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 >> 172.21.15.3:0/2790648531 conn(0x560d5ebd6310 msgr2=0x560d5ec14000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.853+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 shutdown_connections 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.853+0000 7f18a829d740 1 -- 172.21.15.3:0/2790648531 wait complete. 2024-09-12T02:21:50.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.854+0000 7f18a829d740 1 -- start start 2024-09-12T02:21:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.854+0000 7f18a829d740 1 Processor -- start 2024-09-12T02:21:50.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.855+0000 7f18a829d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.855+0000 7f18a829d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec20900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.855+0000 7f18a829d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.855+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec20900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.855+0000 7f189effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec20900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:39414/0 (socket says 172.21.15.3:39414) 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f189effd640 1 -- 172.21.15.3:0/2956934407 learned_addr learned my addr 172.21.15.3:0/2956934407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01bd6e0 con 0x560d5ec0d5d0 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01bd010 con 0x560d5ec0eb40 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18a01ba220 con 0x560d5ec18370 2024-09-12T02:21:50.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f18a829d740 10 client.0 _ll_register_callbacks cb 0x7f1890018ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f189ffff640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.856+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 -- 172.21.15.3:0/2956934407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 msgr2=0x560d5ec20900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec20900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 -- 172.21.15.3:0/2956934407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 msgr2=0x560d5ec203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d5ec12f60 con 0x560d5ec18370 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f189ffff640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec112b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x560d5ec0e070 tx=0x7f189809b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6b25ca3f520a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.857+0000 7f187effd640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.858+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980b8070 con 0x560d5ec18370 2024-09-12T02:21:50.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.858+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f189809b070 con 0x560d5ec18370 2024-09-12T02:21:50.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.858+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1894048070 con 0x560d5ec18370 2024-09-12T02:21:50.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.858+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f189809ffd0 con 0x560d5ec18370 2024-09-12T02:21:50.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.860+0000 7f18a829d740 10 client.25843 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:50.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.860+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560d5ec91e90 con 0x560d5ec18370 2024-09-12T02:21:50.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1894048070 con 0x560d5ec18370 2024-09-12T02:21:50.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f189800f040 con 0x560d5ec18370 2024-09-12T02:21:50.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 10 client.25843 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 10 client.25843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 20 client.25843 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec925c0 0x560d5ec949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 4 client.25843 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 --> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d5eb7ad70 con 0x560d5ec925c0 2024-09-12T02:21:50.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.861+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec925c0 0x560d5ec949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.862+0000 7f187effd640 10 client.25843 ms_handle_connect on v2:172.21.15.92:6834/1180218932 2024-09-12T02:21:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.862+0000 7f189f7fe640 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec925c0 0x560d5ec949e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdd63a8a9fe52f29 server_cookie=16af11a7a4a429d6 in_seq=0 out_seq=0 2024-09-12T02:21:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.863+0000 7f187effd640 1 -- 172.21.15.3:0/2956934407 <== mds.2 v2:172.21.15.92:6834/1180218932 1 ==== command_reply(tid 0: 0 ) ==== 8+0+477 (crc 0 0 0) 0x560d5eb7ad70 con 0x560d5ec925c0 2024-09-12T02:21:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.863+0000 7f187effd640 10 client.25843 handle_command_reply: tid=0 2024-09-12T02:21:50.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.863+0000 7f18a829d740 1 client.25843 shutdown 2024-09-12T02:21:50.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.864+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 msgr2=0x560d5ec112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.864+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec112b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x560d5ec0e070 tx=0x7f189809b670 comp rx=0 tx=0).stop 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.865+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 shutdown_connections 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.865+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:6834/1180218932,v1:172.21.15.92:6835/1180218932] conn(0x560d5ec925c0 0x560d5ec949e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d5ec18370 0x560d5ec112b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d5ec0eb40 0x560d5ec20900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 --2- 172.21.15.3:0/2956934407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d5ec0d5d0 0x560d5ec203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 >> 172.21.15.3:0/2956934407 conn(0x560d5ebd6310 msgr2=0x560d5ec137e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f187effd640 0 client.25843 ms_handle_reset on v2:172.21.15.92:6834/1180218932 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 487917447, 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628409, 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b" 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:21:50.868 INFO:teuthology.orchestra.run.smithi003.stdout: "damage_type": "backtrace", 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4093916290, 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 1099511628453, 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stdout: "path": "/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin" 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 shutdown_connections 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.866+0000 7f18a829d740 1 -- 172.21.15.3:0/2956934407 wait complete. 2024-09-12T02:21:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.867+0000 7f18a829d740 20 client.25843 trim_cache size 0 max 16384 2024-09-12T02:21:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 msgr2=0x7f18880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 0x7f18880a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f18900146d0 tx=0x7f18900099f0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 msgr2=0x7f18a01abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 0x7f18a01abc20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1898009fd0 tx=0x7f189800edf0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 shutdown_connections 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f18880a2c00 0x7f18880a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f18a011b140 0x7f18a01abc20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a0109830 0x7f18a01a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 --2- 172.21.15.3:0/2230903855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18a0109450 0x7f18a01a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.869+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 >> 172.21.15.3:0/2230903855 conn(0x7f18a006e140 msgr2=0x7f18a011bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.870+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 shutdown_connections 2024-09-12T02:21:50.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:21:50.870+0000 7f18a5ec4640 1 -- 172.21.15.3:0/2230903855 wait complete. 2024-09-12T02:21:51.661 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd6859252', total seconds elapsed: 89.0 2024-09-12T02:21:51.733 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:51 smithi003 ceph-mon[31476]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 3.83k op/s 2024-09-12T02:21:51.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 -- 172.21.15.3:0/3574764659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18102010 msgr2=0x7f1c18102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/3574764659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18102010 0x7f1c18102410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009860 tx=0x7f1c0802a340 comp rx=0 tx=0).stop 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 -- 172.21.15.3:0/3574764659 shutdown_connections 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/3574764659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c181015d0 0x7f1c18108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/3574764659 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c18100c10 0x7f1c18101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/3574764659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18102010 0x7f1c18102410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 -- 172.21.15.3:0/3574764659 >> 172.21.15.3:0/3574764659 conn(0x7f1c180fb930 msgr2=0x7f1c180fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 -- 172.21.15.3:0/3574764659 shutdown_connections 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.877+0000 7f1c1e3af640 1 -- 172.21.15.3:0/3574764659 wait complete. 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 Processor -- start 2024-09-12T02:21:51.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 -- start start 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18100c10 0x7f1c18197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c18102010 0x7f1c1819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810c690 con 0x7f1c181015d0 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810c510 con 0x7f1c18102010 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c1e3af640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1810c810 con 0x7f1c18100c10 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39144/0 (socket says 172.21.15.3:39144) 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.878+0000 7f1c177fe640 1 -- 172.21.15.3:0/2913359158 learned_addr learned my addr 172.21.15.3:0/2913359158 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 -- 172.21.15.3:0/2913359158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18100c10 msgr2=0x7f1c18197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18100c10 0x7f1c18197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 -- 172.21.15.3:0/2913359158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c18102010 msgr2=0x7f1c1819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c18102010 0x7f1c1819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 -- 172.21.15.3:0/2913359158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1819fab0 con 0x7f1c181015d0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00e9c0 tx=0x7f1c0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c157fa640 1 -- 172.21.15.3:0/2913359158 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00cde0 con 0x7f1c181015d0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1819fda0 con 0x7f1c181015d0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c157fa640 1 -- 172.21.15.3:0/2913359158 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c0035f0 con 0x7f1c181015d0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c157fa640 1 -- 172.21.15.3:0/2913359158 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c010690 con 0x7f1c181015d0 2024-09-12T02:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.879+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a01b0 con 0x7f1c181015d0 2024-09-12T02:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.882+0000 7f1c21224740 1 Processor -- start 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 -- start start 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda077bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda077c370 0x55cda077e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cda06da2f0 con 0x55cda07769a0 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cda0682270 con 0x55cda077c370 2024-09-12T02:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c21224740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cda06d62b0 con 0x55cda07772e0 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c1c94c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c157fa640 1 -- 172.21.15.3:0/2913359158 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1c0c0037a0 con 0x7f1c181015d0 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c1c94c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39154/0 (socket says 172.21.15.3:39154) 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c1c94c640 1 -- 172.21.15.3:0/2094934249 learned_addr learned my addr 172.21.15.3:0/2094934249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda077bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.883+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda077c370 0x55cda077e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 -- 172.21.15.3:0/2094934249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 msgr2=0x55cda077bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda077bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 -- 172.21.15.3:0/2094934249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda077c370 msgr2=0x55cda077e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda077c370 0x55cda077e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 -- 172.21.15.3:0/2094934249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cda0681340 con 0x55cda07769a0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c1c94c640 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ea0 tx=0x7f1c10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf8afc417d623b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c157fa640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1bfe7fc640 1 -- 172.21.15.3:0/2094934249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10004d20 con 0x55cda07769a0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c157fa640 1 -- 172.21.15.3:0/2913359158 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1c0c0cf590 con 0x7f1c181015d0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1bfe7fc640 1 -- 172.21.15.3:0/2094934249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1000ee60 con 0x55cda07769a0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.884+0000 7f1bfe7fc640 1 -- 172.21.15.3:0/2094934249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c100107f0 con 0x55cda07769a0 2024-09-12T02:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 -- 172.21.15.3:0/2094934249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 msgr2=0x55cda0776da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 0x7f1c000a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55cda0777d10 tx=0x7f1c080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ea0 tx=0x7f1c10009f90 comp rx=0 tx=0).stop 2024-09-12T02:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 -- 172.21.15.3:0/2094934249 shutdown_connections 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda077c370 0x55cda077e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda077bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 --2- 172.21.15.3:0/2094934249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0776da0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 -- 172.21.15.3:0/2094934249 >> 172.21.15.3:0/2094934249 conn(0x55cda0772010 msgr2=0x55cda0772410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 -- 172.21.15.3:0/2094934249 shutdown_connections 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.885+0000 7f1c21224740 1 -- 172.21.15.3:0/2094934249 wait complete. 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 -- start start 2024-09-12T02:21:51.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 Processor -- start 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07769a0 0x55cda0780c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda077c370 0x55cda07881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55cda077f390 con 0x55cda077c370 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55cda0663290 con 0x55cda07769a0 2024-09-12T02:21:51.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cda077f510 con 0x55cda07772e0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.886+0000 7f1c21224740 10 client.0 _ll_register_callbacks cb 0x55cda0694140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c1c94c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda077c370 0x55cda07881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56888/0 (socket says 172.21.15.3:56888) 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 -- 172.21.15.3:0/4088417407 learned_addr learned my addr 172.21.15.3:0/4088417407 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c17fff640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07769a0 0x55cda0780c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 -- 172.21.15.3:0/4088417407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07769a0 msgr2=0x55cda0780c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07769a0 0x55cda0780c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 -- 172.21.15.3:0/4088417407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda077c370 msgr2=0x55cda07881f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda077c370 0x55cda07881f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 -- 172.21.15.3:0/4088417407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cda03ed210 con 0x55cda07772e0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1c177fe640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c09bae0 tx=0x7f1c0c09ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ecb91a89261f718 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.887+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c09bde0 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1000ee60 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1c0c0173d0 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c0a0190 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 10 client.25852 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cda04b9620 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1000ee60 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1c0c09b230 con 0x55cda07772e0 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 10 client.25852 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 20 client.25852 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda078c990 0x55cda078ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 4 client.25852 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:21:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.888+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cda06e7a20 con 0x55cda078c990 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.889+0000 7f1c17fff640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda078c990 0x55cda078ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.889+0000 7f1c17fff640 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda078c990 0x55cda078ed80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00d124914266c78 server_cookie=91a0ad0450fdb1b9 in_seq=0 out_seq=0 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.889+0000 7f1bf77fe640 10 client.25852 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.890+0000 7f1bf77fe640 1 -- 172.21.15.3:0/4088417407 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cda06e7a20 con 0x55cda078c990 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.890+0000 7f1bf77fe640 10 client.25852 handle_command_reply: tid=0 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.890+0000 7f1c21224740 1 client.25852 shutdown 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 msgr2=0x55cda07811b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c09bae0 tx=0x7f1c0c09ba20 comp rx=0 tx=0).stop 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 shutdown_connections 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda078c990 0x55cda078ed80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda077c370 0x55cda07881f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda07772e0 0x55cda07811b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 --2- 172.21.15.3:0/4088417407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07769a0 0x55cda0780c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 >> 172.21.15.3:0/4088417407 conn(0x55cda0772010 msgr2=0x55cda0773820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1bf77fe640 0 client.25852 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 shutdown_connections 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 1 -- 172.21.15.3:0/4088417407 wait complete. 2024-09-12T02:21:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.891+0000 7f1c21224740 20 client.25852 trim_cache size 0 max 16384 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.916+0000 7f1c21224740 1 Processor -- start 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 -- start start 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0773400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda07881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0773940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ab440 con 0x55cda07769a0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ad040 con 0x55cda07772e0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c21224740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181abb10 con 0x55cda0772010 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0773400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda07881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0773400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56904/0 (socket says 172.21.15.3:56904) 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c17fff640 1 -- 172.21.15.3:0/84127067 learned_addr learned my addr 172.21.15.3:0/84127067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c1c94c640 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0773940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 -- 172.21.15.3:0/84127067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 msgr2=0x55cda0773400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0773400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 -- 172.21.15.3:0/84127067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 msgr2=0x55cda0773940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0773940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.917+0000 7f1c177fe640 1 -- 172.21.15.3:0/84127067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cda0773e80 con 0x55cda07769a0 2024-09-12T02:21:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.918+0000 7f1c177fe640 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda07881b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c09ba90 tx=0x7f1c0c0d2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d3f215b22850ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.918+0000 7f1bfdffb640 1 -- 172.21.15.3:0/84127067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c09bde0 con 0x55cda07769a0 2024-09-12T02:21:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.918+0000 7f1bfdffb640 1 -- 172.21.15.3:0/84127067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c009530 con 0x55cda07769a0 2024-09-12T02:21:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.918+0000 7f1bfdffb640 1 -- 172.21.15.3:0/84127067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0cf0d0 con 0x55cda07769a0 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 -- 172.21.15.3:0/84127067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 msgr2=0x55cda07881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda07881b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c09ba90 tx=0x7f1c0c0d2e40 comp rx=0 tx=0).stop 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 -- 172.21.15.3:0/84127067 shutdown_connections 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0773940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda07881b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 --2- 172.21.15.3:0/84127067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0773400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 -- 172.21.15.3:0/84127067 >> 172.21.15.3:0/84127067 conn(0x55cda077bec0 msgr2=0x55cda07711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 -- 172.21.15.3:0/84127067 shutdown_connections 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.919+0000 7f1c21224740 1 -- 172.21.15.3:0/84127067 wait complete. 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.920+0000 7f1c21224740 1 -- start start 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.920+0000 7f1c21224740 1 Processor -- start 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0785c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0787150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ac8e0 con 0x55cda07769a0 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ac210 con 0x55cda07772e0 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181a9420 con 0x55cda0772010 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c21224740 10 client.0 _ll_register_callbacks cb 0x7f1c0c09bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39176/0 (socket says 172.21.15.3:39176) 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c177fe640 1 -- 172.21.15.3:0/2264838280 learned_addr learned my addr 172.21.15.3:0/2264838280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c1c94c640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0787150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.921+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0785c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 -- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 msgr2=0x55cda0785c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0785c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 -- 172.21.15.3:0/2264838280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 msgr2=0x55cda0787150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0787150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 -- 172.21.15.3:0/2264838280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cda0787760 con 0x55cda07769a0 2024-09-12T02:21:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c177fe640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0cd3f0 tx=0x7f1c0c017760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816915ae2e7a9e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1bf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c09bde0 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1c0c0cf0d0 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cda06e1d70 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c0a4a50 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c21224740 10 client.16269 fetch_fsmap learned FSMap version 34 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.922+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cda07f79d0 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cda06e1d70 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1c0c0baa20 con 0x55cda07769a0 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 10 client.16269 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 20 client.16269 populate_metadata read hostname 'smithi003' 2024-09-12T02:21:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda077c370 0x55cda07fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 4 client.16269 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cda065e900 con 0x55cda077c370 2024-09-12T02:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.923+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda077c370 0x55cda07fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.924+0000 7f1bf77fe640 10 client.16269 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:51.924+0000 7f1c17fff640 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda077c370 0x55cda07fa1b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a433dd3a81359808 server_cookie=5b668243e04bd802 in_seq=0 out_seq=0 2024-09-12T02:21:52.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:51 smithi026 ceph-mon[35496]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 3.83k op/s 2024-09-12T02:21:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:51 smithi092 ceph-mon[36089]: pgmap v440: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 20 MiB/s wr, 3.83k op/s 2024-09-12T02:21:54.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:53 smithi026 ceph-mon[35496]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 4.51k op/s 2024-09-12T02:21:54.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:53 smithi003 ceph-mon[31476]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 4.51k op/s 2024-09-12T02:21:54.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:53 smithi092 ceph-mon[36089]: pgmap v441: 97 pgs: 97 active+clean; 13 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 23 MiB/s wr, 4.51k op/s 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.419+0000 7f1bf77fe640 1 -- 172.21.15.3:0/2264838280 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cda065e900 con 0x55cda077c370 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.419+0000 7f1bf77fe640 10 client.16269 handle_command_reply: tid=0 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.419+0000 7f1c21224740 1 client.16269 shutdown 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.419+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 msgr2=0x55cda0786140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.419+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0cd3f0 tx=0x7f1c0c017760 comp rx=0 tx=0).stop 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 shutdown_connections 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55cda077c370 0x55cda07fa1b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cda07772e0 0x55cda0787150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55cda07769a0 0x55cda0786140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 --2- 172.21.15.3:0/2264838280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cda0772010 0x55cda0785c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 >> 172.21.15.3:0/2264838280 conn(0x55cda077bec0 msgr2=0x55cda07723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1bf77fe640 0 client.16269 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:21:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 shutdown_connections 2024-09-12T02:21:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 1 -- 172.21.15.3:0/2264838280 wait complete. 2024-09-12T02:21:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.420+0000 7f1c21224740 20 client.16269 trim_cache size 0 max 16384 2024-09-12T02:21:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:21:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:21:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:21:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 msgr2=0x7f1c000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 0x7f1c000a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55cda0777d10 tx=0x7f1c080058d0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 msgr2=0x7f1c181983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00e9c0 tx=0x7f1c0c00ee80 comp rx=0 tx=0).stop 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 shutdown_connections 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c18102010 0x7f1c1819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c181015d0 0x7f1c181983d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 --2- 172.21.15.3:0/2913359158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c18100c10 0x7f1c18197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:21:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 >> 172.21.15.3:0/2913359158 conn(0x7f1c180fb930 msgr2=0x7f1c180ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:21:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 shutdown_connections 2024-09-12T02:21:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:21:54.421+0000 7f1c1e3af640 1 -- 172.21.15.3:0/2913359158 wait complete. 2024-09-12T02:21:56.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:55 smithi026 ceph-mon[35496]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 MiB/s wr, 3.75k op/s 2024-09-12T02:21:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:55 smithi003 ceph-mon[31476]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 MiB/s wr, 3.75k op/s 2024-09-12T02:21:56.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:55 smithi092 ceph-mon[36089]: pgmap v442: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 19 MiB/s wr, 3.75k op/s 2024-09-12T02:21:57.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:56 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:56 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:56 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:58.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:57 smithi003 ceph-mon[31476]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 47 MiB/s wr, 8.70k op/s 2024-09-12T02:21:58.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:57 smithi092 ceph-mon[36089]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 47 MiB/s wr, 8.70k op/s 2024-09-12T02:21:58.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:57 smithi026 ceph-mon[35496]: pgmap v443: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 47 MiB/s wr, 8.70k op/s 2024-09-12T02:21:59.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:58 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:59.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:58 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:21:59.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:58 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:22:00.038 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:21:59 smithi092 ceph-mon[36089]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 57 MiB/s wr, 10.22k op/s 2024-09-12T02:22:00.101 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:21:59 smithi003 ceph-mon[31476]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 57 MiB/s wr, 10.22k op/s 2024-09-12T02:22:00.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:22:00.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:21:59 smithi026 ceph-mon[35496]: pgmap v444: 97 pgs: 97 active+clean; 13 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 57 MiB/s wr, 10.22k op/s 2024-09-12T02:22:00.742 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:22:02.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:01 smithi092 ceph-mon[36089]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 74 MiB/s wr, 12.73k op/s 2024-09-12T02:22:02.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:01 smithi026 ceph-mon[35496]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 74 MiB/s wr, 12.73k op/s 2024-09-12T02:22:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:01 smithi003 ceph-mon[31476]: pgmap v445: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 74 MiB/s wr, 12.73k op/s 2024-09-12T02:22:04.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:03 smithi026 ceph-mon[35496]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 97 MiB/s wr, 16.19k op/s 2024-09-12T02:22:04.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:03 smithi003 ceph-mon[31476]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 97 MiB/s wr, 16.19k op/s 2024-09-12T02:22:04.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:03 smithi092 ceph-mon[36089]: pgmap v446: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 97 MiB/s wr, 16.19k op/s 2024-09-12T02:22:05.934 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:05 smithi003 ceph-mon[31476]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 99 MiB/s wr, 15.86k op/s 2024-09-12T02:22:06.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:05 smithi026 ceph-mon[35496]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 99 MiB/s wr, 15.86k op/s 2024-09-12T02:22:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:05 smithi092 ceph-mon[36089]: pgmap v447: 97 pgs: 97 active+clean; 13 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 99 MiB/s wr, 15.86k op/s 2024-09-12T02:22:07.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 -- 172.21.15.3:0/3540648340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e81012d0 msgr2=0x7fb8e8104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3540648340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e81012d0 0x7fb8e8104e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800a970 tx=0x7fb8d802b670 comp rx=0 tx=0).stop 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 -- 172.21.15.3:0/3540648340 shutdown_connections 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3540648340 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81055a0 0x7fb8e810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3540648340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e81012d0 0x7fb8e8104e70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3540648340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e8100920 0x7fb8e8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 -- 172.21.15.3:0/3540648340 >> 172.21.15.3:0/3540648340 conn(0x7fb8e8075f70 msgr2=0x7fb8e8076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 -- 172.21.15.3:0/3540648340 shutdown_connections 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.928+0000 7fb8eca30640 1 -- 172.21.15.3:0/3540648340 wait complete. 2024-09-12T02:22:07.215 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 Processor -- start 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 -- start start 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e81055a0 0x7fb8e8196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810fd80 con 0x7fb8e81012d0 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810fc00 con 0x7fb8e8100920 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8eca30640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e810ff00 con 0x7fb8e81055a0 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8e659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55562/0 (socket says 172.21.15.3:55562) 2024-09-12T02:22:07.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8e659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:43174/0 (socket says 172.21.15.3:43174) 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.930+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/3608131113 learned_addr learned my addr 172.21.15.3:0/3608131113 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/3608131113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e81055a0 msgr2=0x7fb8e8196690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e81055a0 0x7fb8e8196690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/3608131113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 msgr2=0x7fb8e819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/3608131113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e8196d00 con 0x7fb8e81012d0 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e659c640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.931+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8032630 tx=0x7fb8d8032660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.932+0000 7fb8d77fe640 1 -- 172.21.15.3:0/3608131113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8040070 con 0x7fb8e81012d0 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.932+0000 7fb8d77fe640 1 -- 172.21.15.3:0/3608131113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8d8032a10 con 0x7fb8e81012d0 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.932+0000 7fb8d77fe640 1 -- 172.21.15.3:0/3608131113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8045460 con 0x7fb8e81012d0 2024-09-12T02:22:07.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.934+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e8196f90 con 0x7fb8e81012d0 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.934+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e819ae80 con 0x7fb8e81012d0 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.941+0000 7fb8eee09740 1 Processor -- start 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.942+0000 7fb8eee09740 1 -- start start 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.942+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.942+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.942+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8eee09740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562905a752f0 con 0x562905b1a370 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8eee09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562905a1ccf0 con 0x562905b10b40 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8eee09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562905a712b0 con 0x562905b0f5d0 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:43186/0 (socket says 172.21.15.3:43186) 2024-09-12T02:22:07.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8d77fe640 1 -- 172.21.15.3:0/3608131113 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb8d8004750 con 0x7fb8e81012d0 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 -- 172.21.15.3:0/3249002359 learned_addr learned my addr 172.21.15.3:0/3249002359 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e659c640 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 -- 172.21.15.3:0/3249002359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 msgr2=0x562905b0fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b0fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 -- 172.21.15.3:0/3249002359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 msgr2=0x562905b1c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b1c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.943+0000 7fb8e6d9d640 1 -- 172.21.15.3:0/3249002359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562905a013a0 con 0x562905b10b40 2024-09-12T02:22:07.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.944+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009f80 tx=0x7fb8e000e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42a3208abcf95f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.945+0000 7fb8d77fe640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 0x7fb8d00a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.945+0000 7fb8d77fe640 1 -- 172.21.15.3:0/3608131113 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8d80f0900 con 0x7fb8e81012d0 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.945+0000 7fb8e659c640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 0x7fb8d00a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.944+0000 7fb8d4ff9640 1 -- 172.21.15.3:0/3249002359 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004d20 con 0x562905b10b40 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.946+0000 7fb8d4ff9640 1 -- 172.21.15.3:0/3249002359 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e0003e50 con 0x562905b10b40 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.947+0000 7fb8eee09740 1 -- 172.21.15.3:0/3249002359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 msgr2=0x562905b0f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.947+0000 7fb8eee09740 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009f80 tx=0x7fb8e000e880 comp rx=0 tx=0).stop 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.947+0000 7fb8eee09740 1 -- 172.21.15.3:0/3249002359 shutdown_connections 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.948+0000 7fb8eee09740 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.948+0000 7fb8eee09740 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.948+0000 7fb8eee09740 1 --2- 172.21.15.3:0/3249002359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b0f090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.948+0000 7fb8eee09740 1 -- 172.21.15.3:0/3249002359 >> 172.21.15.3:0/3249002359 conn(0x562905ad8310 msgr2=0x562905b0bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.951+0000 7fb8e659c640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 0x7fb8d00a5290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562905b0cac0 tx=0x7fb8dc00c330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.951+0000 7fb8eee09740 1 -- 172.21.15.3:0/3249002359 shutdown_connections 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.952+0000 7fb8eee09740 1 -- 172.21.15.3:0/3249002359 wait complete. 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.952+0000 7fb8eee09740 1 -- start start 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.953+0000 7fb8eee09740 1 Processor -- start 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.953+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b0f5d0 0x562905b19c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.953+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.953+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b1a370 0x562905b17450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8eee09740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562905b2e0d0 con 0x562905b10b40 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8eee09740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x562905a30060 con 0x562905b0f5d0 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8eee09740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562905b2e250 con 0x562905b1a370 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55574/0 (socket says 172.21.15.3:55574) 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/2662397489 learned_addr learned my addr 172.21.15.3:0/2662397489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.954+0000 7fb8eee09740 10 client.0 _ll_register_callbacks cb 0x5629057838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b1a370 0x562905b17450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b1a370 msgr2=0x562905b17450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b1a370 0x562905b17450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/2662397489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b0f5d0 msgr2=0x562905b19c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:22:07.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b0f5d0 0x562905b19c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/2662397489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629057e63c0 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80bd080 tx=0x7fb8d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1fecf1957e2bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.955+0000 7fb8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.957+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8030d90 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.957+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8d80c1120 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.959+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e0003e50 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.962+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8d803ae60 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.963+0000 7fb8eee09740 10 client.16281 fetch_fsmap learned FSMap version 36 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.963+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562905b39b30 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e0003e50 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fb8d80d7050 con 0x562905b10b40 2024-09-12T02:22:07.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 10 client.16281 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 10 client.16281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 20 client.16281 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905b3af90 0x562905b3d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 4 client.16281 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.966+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562905a82860 con 0x562905b3af90 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.967+0000 7fb8e659c640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905b3af90 0x562905b3d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.970+0000 7fb8cdffb640 10 client.16281 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.970+0000 7fb8e659c640 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905b3af90 0x562905b3d410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d99f7ec1860725a server_cookie=f370bc460ef20a14 in_seq=0 out_seq=0 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.971+0000 7fb8cdffb640 1 -- 172.21.15.3:0/2662397489 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562905a82860 con 0x562905b3af90 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.971+0000 7fb8cdffb640 10 client.16281 handle_command_reply: tid=0 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.971+0000 7fb8eee09740 1 client.16281 shutdown 2024-09-12T02:22:07.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.971+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 msgr2=0x562905b16f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.971+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80bd080 tx=0x7fb8d80046c0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 shutdown_connections 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905b3af90 0x562905b3d410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8cdffb640 0 client.16281 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b1a370 0x562905b17450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b10b40 0x562905b16f10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 --2- 172.21.15.3:0/2662397489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b0f5d0 0x562905b19c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.972+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 >> 172.21.15.3:0/2662397489 conn(0x562905ad8310 msgr2=0x562905b11960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.977+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 shutdown_connections 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.978+0000 7fb8eee09740 1 -- 172.21.15.3:0/2662397489 wait complete. 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:05.978+0000 7fb8eee09740 20 client.16281 trim_cache size 0 max 16384 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.003+0000 7fb8eee09740 1 Processor -- start 2024-09-12T02:22:07.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.003+0000 7fb8eee09740 1 -- start start 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.003+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b12120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b12660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49412/0 (socket says 172.21.15.3:49412) 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e659c640 1 -- 172.21.15.3:0/328592056 learned_addr learned my addr 172.21.15.3:0/328592056 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8073650 con 0x562905b1a370 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b5820 con 0x562905b10b40 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8073d20 con 0x562905b0f5d0 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b12660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e659c640 1 -- 172.21.15.3:0/328592056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 msgr2=0x562905b12120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.004+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b12120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.005+0000 7fb8e659c640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b12120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.005+0000 7fb8e659c640 1 -- 172.21.15.3:0/328592056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 msgr2=0x562905b12660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.005+0000 7fb8e659c640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b12660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.005+0000 7fb8e659c640 1 -- 172.21.15.3:0/328592056 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562905b19810 con 0x562905b0f5d0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.005+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b12660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.006+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b12120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.006+0000 7fb8e659c640 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562905b3dc20 tx=0x7fb8dc013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1477eb494810f665 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.006+0000 7fb8cffff640 1 -- 172.21.15.3:0/328592056 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011c60 con 0x562905b0f5d0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.006+0000 7fb8cffff640 1 -- 172.21.15.3:0/328592056 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8dc011e00 con 0x562905b0f5d0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.006+0000 7fb8cffff640 1 -- 172.21.15.3:0/328592056 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc003890 con 0x562905b0f5d0 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 msgr2=0x562905b37420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x562905b3dc20 tx=0x7fb8dc013000 comp rx=0 tx=0).stop 2024-09-12T02:22:07.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 shutdown_connections 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b1a370 0x562905b12660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b10b40 0x562905b12120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 --2- 172.21.15.3:0/328592056 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b0f5d0 0x562905b37420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.008+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 >> 172.21.15.3:0/328592056 conn(0x562905ad8310 msgr2=0x562905b18f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.009+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 shutdown_connections 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.009+0000 7fb8eee09740 1 -- 172.21.15.3:0/328592056 wait complete. 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.010+0000 7fb8eee09740 1 -- start start 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.010+0000 7fb8eee09740 1 Processor -- start 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.011+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 0x562905b33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.011+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.011+0000 7fb8eee09740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e5d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49424/0 (socket says 172.21.15.3:49424) 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:43216/0 (socket says 172.21.15.3:43216) 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/1264874490 learned_addr learned my addr 172.21.15.3:0/1264874490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8e659c640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 0x562905b33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b5120 con 0x562905b0f5d0 2024-09-12T02:22:07.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e81b4a80 con 0x562905b1a370 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e8071630 con 0x562905b10b40 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.012+0000 7fb8eee09740 10 client.0 _ll_register_callbacks cb 0x7fb8d800a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/1264874490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 msgr2=0x562905b2ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/1264874490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 msgr2=0x562905b33860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 0x562905b33860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562905b2f420 con 0x562905b10b40 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e659c640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 0x562905b33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e6d9d640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8e5d9b640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d80c5b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186589c202d872c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:22:07.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8030d90 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb8d80b8e40 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.013+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8dc011e00 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.016+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8d80bd7f0 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.018+0000 7fb8eee09740 10 client.25867 fetch_fsmap learned FSMap version 36 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.018+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562905ba5740 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8dc011e00 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7fb8d80d7020 con 0x562905b10b40 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 10 client.25867 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 10 client.25867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:22:07.231 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 20 client.25867 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905ba5d80 0x562905ba81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 4 client.25867 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 --> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562905a7cd70 con 0x562905ba5d80 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.020+0000 7fb8e659c640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905ba5d80 0x562905ba81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.021+0000 7fb8e659c640 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905ba5d80 0x562905ba81a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d197946926cd370 server_cookie=8fff8905e2fe0cfc in_seq=0 out_seq=0 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.021+0000 7fb8cdffb640 10 client.25867 ms_handle_connect on v2:172.21.15.92:6836/487477954 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.022+0000 7fb8cdffb640 1 -- 172.21.15.3:0/1264874490 <== mds.1 v2:172.21.15.92:6836/487477954 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562905a7cd70 con 0x562905ba5d80 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.022+0000 7fb8cdffb640 10 client.25867 handle_command_reply: tid=0 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.022+0000 7fb8eee09740 1 client.25867 shutdown 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.028+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 msgr2=0x562905b33da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.028+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8000c00 tx=0x7fb8d80c5b50 comp rx=0 tx=0).stop 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 shutdown_connections 2024-09-12T02:22:07.232 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:6836/487477954,v1:172.21.15.92:6837/487477954] conn(0x562905ba5d80 0x562905ba81a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562905b1a370 0x562905b2ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562905b10b40 0x562905b33da0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 --2- 172.21.15.3:0/1264874490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562905b0f5d0 0x562905b33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 >> 172.21.15.3:0/1264874490 conn(0x562905ad8310 msgr2=0x562905b167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.029+0000 7fb8cdffb640 0 client.25867 ms_handle_reset on v2:172.21.15.92:6836/487477954 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.030+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 shutdown_connections 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.030+0000 7fb8eee09740 1 -- 172.21.15.3:0/1264874490 wait complete. 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.030+0000 7fb8eee09740 20 client.25867 trim_cache size 0 max 16384 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.038+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 msgr2=0x7fb8d00a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.038+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 0x7fb8d00a5290 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562905b0cac0 tx=0x7fb8dc00c330 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.038+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 msgr2=0x7fb8e8196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.038+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8032630 tx=0x7fb8d8032660 comp rx=0 tx=0).stop 2024-09-12T02:22:07.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 shutdown_connections 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb8d00a2dd0 0x7fb8d00a5290 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e81055a0 0x7fb8e8196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb8e81012d0 0x7fb8e8196150 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 --2- 172.21.15.3:0/3608131113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8e8100920 0x7fb8e819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 >> 172.21.15.3:0/3608131113 conn(0x7fb8e8075f70 msgr2=0x7fb8e80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 shutdown_connections 2024-09-12T02:22:07.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:06.039+0000 7fb8eca30640 1 -- 172.21.15.3:0/3608131113 wait complete. 2024-09-12T02:22:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.577+0000 7f45015f8640 1 -- 172.21.15.3:0/809305801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc100a40 msgr2=0x7f44fc100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.577+0000 7f45015f8640 1 --2- 172.21.15.3:0/809305801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc100a40 0x7f44fc100e40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009900 tx=0x7f44f002a420 comp rx=0 tx=0).stop 2024-09-12T02:22:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.581+0000 7f45015f8640 1 -- 172.21.15.3:0/809305801 shutdown_connections 2024-09-12T02:22:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.581+0000 7f45015f8640 1 --2- 172.21.15.3:0/809305801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc1025f0 0x7f44fc069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.581+0000 7f45015f8640 1 --2- 172.21.15.3:0/809305801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc101c30 0x7f44fc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.581+0000 7f45015f8640 1 --2- 172.21.15.3:0/809305801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc100a40 0x7f44fc100e40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.581+0000 7f45015f8640 1 -- 172.21.15.3:0/809305801 >> 172.21.15.3:0/809305801 conn(0x7f44fc0fc1c0 msgr2=0x7f44fc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.583+0000 7f45015f8640 1 -- 172.21.15.3:0/809305801 shutdown_connections 2024-09-12T02:22:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.584+0000 7f45015f8640 1 -- 172.21.15.3:0/809305801 wait complete. 2024-09-12T02:22:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.585+0000 7f45015f8640 1 Processor -- start 2024-09-12T02:22:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.586+0000 7f45015f8640 1 -- start start 2024-09-12T02:22:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc101c30 0x7f44fc0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc1025f0 0x7f44fc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06c220 con 0x7f44fc1025f0 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06c0a0 con 0x7f44fc101c30 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f45015f8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc06c3a0 con 0x7f44fc100a40 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49450/0 (socket says 172.21.15.3:49450) 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.587+0000 7f44faffd640 1 -- 172.21.15.3:0/2544412962 learned_addr learned my addr 172.21.15.3:0/2544412962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 -- 172.21.15.3:0/2544412962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc101c30 msgr2=0x7f44fc0720f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc101c30 0x7f44fc0720f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 -- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc1025f0 msgr2=0x7f44fc073130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc1025f0 0x7f44fc073130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 -- 172.21.15.3:0/2544412962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc10f1e0 con 0x7f44fc100a40 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44faffd640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44f0031610 tx=0x7f44f0032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.588+0000 7f44e3fff640 1 -- 172.21.15.3:0/2544412962 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f003f030 con 0x7f44fc100a40 2024-09-12T02:22:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.589+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc073670 con 0x7f44fc100a40 2024-09-12T02:22:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.589+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc073bb0 con 0x7f44fc100a40 2024-09-12T02:22:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.589+0000 7f44e3fff640 1 -- 172.21.15.3:0/2544412962 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0003ad0 con 0x7f44fc100a40 2024-09-12T02:22:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.589+0000 7f44e3fff640 1 -- 172.21.15.3:0/2544412962 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0035070 con 0x7f44fc100a40 2024-09-12T02:22:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.592+0000 7f4504468740 1 Processor -- start 2024-09-12T02:22:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.592+0000 7f4504468740 1 -- start start 2024-09-12T02:22:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36cd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cd72e0 0x557c36cdbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557c36c3a2f0 con 0x557c36cd72e0 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c36be2270 con 0x557c36cd69a0 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.594+0000 7f4504468740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557c36c362b0 con 0x557c36cdc370 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.595+0000 7f44fa7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.595+0000 7f44fa7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49454/0 (socket says 172.21.15.3:49454) 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.595+0000 7f44fa7fc640 1 -- 172.21.15.3:0/84506307 learned_addr learned my addr 172.21.15.3:0/84506307 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 -- 172.21.15.3:0/84506307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 msgr2=0x557c36cd6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36cd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 -- 172.21.15.3:0/84506307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cd72e0 msgr2=0x557c36cdbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cd72e0 0x557c36cdbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 -- 172.21.15.3:0/84506307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c36be1340 con 0x557c36cdc370 2024-09-12T02:22:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44e3fff640 1 -- 172.21.15.3:0/2544412962 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f44f0042030 con 0x7f44fc100a40 2024-09-12T02:22:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.596+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d70 tx=0x7f44ec00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c712dcf7b289e2bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.597+0000 7f44e3fff640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 0x7f44e40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.598+0000 7f44e3fff640 1 -- 172.21.15.3:0/2544412962 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f44f00ef5f0 con 0x7f44fc100a40 2024-09-12T02:22:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.599+0000 7f44e17fa640 1 -- 172.21.15.3:0/84506307 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00bd40 con 0x557c36cdc370 2024-09-12T02:22:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.599+0000 7f44e17fa640 1 -- 172.21.15.3:0/84506307 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44ec017070 con 0x557c36cdc370 2024-09-12T02:22:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.600+0000 7f4504468740 1 -- 172.21.15.3:0/84506307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 msgr2=0x557c36cde830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.600+0000 7f4504468740 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d70 tx=0x7f44ec00b420 comp rx=0 tx=0).stop 2024-09-12T02:22:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.599+0000 7f44e17fa640 1 -- 172.21.15.3:0/84506307 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec011710 con 0x557c36cdc370 2024-09-12T02:22:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.604+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 0x7f44e40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.605+0000 7f4504468740 1 -- 172.21.15.3:0/84506307 shutdown_connections 2024-09-12T02:22:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.605+0000 7f4504468740 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cdc370 0x557c36cde830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.605+0000 7f4504468740 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cd72e0 0x557c36cdbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.605+0000 7f4504468740 1 --2- 172.21.15.3:0/84506307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36cd6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.605+0000 7f4504468740 1 -- 172.21.15.3:0/84506307 >> 172.21.15.3:0/84506307 conn(0x557c36cd2010 msgr2=0x557c36cd2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.609+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 0x7f44e40a5250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44f40058c0 tx=0x7f44f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.612+0000 7f4504468740 1 -- 172.21.15.3:0/84506307 shutdown_connections 2024-09-12T02:22:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.614+0000 7f4504468740 1 -- 172.21.15.3:0/84506307 wait complete. 2024-09-12T02:22:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.615+0000 7f4504468740 1 -- start start 2024-09-12T02:22:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.615+0000 7f4504468740 1 Processor -- start 2024-09-12T02:22:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557c36cdf390 con 0x557c36cdc370 2024-09-12T02:22:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x557c36bc3290 con 0x557c36cd72e0 2024-09-12T02:22:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557c36cdf510 con 0x557c36cd69a0 2024-09-12T02:22:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f4504468740 10 client.0 _ll_register_callbacks cb 0x557c36bf4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49462/0 (socket says 172.21.15.3:49462) 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f44faffd640 1 -- 172.21.15.3:0/3853907360 learned_addr learned my addr 172.21.15.3:0/3853907360 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.616+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 -- 172.21.15.3:0/3853907360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 msgr2=0x557c36ce11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 -- 172.21.15.3:0/3853907360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 msgr2=0x557c36ce81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 -- 172.21.15.3:0/3853907360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3694d210 con 0x557c36cd69a0 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:22:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.617+0000 7f44faffd640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44f00bbae0 tx=0x7f44f00bb800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f82b963c0a8875a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.619+0000 7f44deff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:22:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.619+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00bf070 con 0x557c36cd69a0 2024-09-12T02:22:07.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.619+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f44f00b83c0 con 0x557c36cd69a0 2024-09-12T02:22:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.621+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44ec017070 con 0x557c36cd69a0 2024-09-12T02:22:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.623+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44f00bc4e0 con 0x557c36cd69a0 2024-09-12T02:22:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.625+0000 7f4504468740 10 client.25876 fetch_fsmap learned FSMap version 36 2024-09-12T02:22:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.625+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c36a19620 con 0x557c36cd69a0 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.628+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44ec017070 con 0x557c36cd69a0 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.628+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f44f00c45c0 con 0x557c36cd69a0 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 10 client.25876 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 20 client.25876 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36cfd970 0x557c36cffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 4 client.25876 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.629+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c36c47a20 con 0x557c36cfd970 2024-09-12T02:22:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.630+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36cfd970 0x557c36cffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.634+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36cfd970 0x557c36cffdb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6247908b08500b server_cookie=7e857f026296c83f in_seq=0 out_seq=0 2024-09-12T02:22:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.638+0000 7f44deff5640 10 client.25876 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.639+0000 7f44deff5640 1 -- 172.21.15.3:0/3853907360 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c36c47a20 con 0x557c36cfd970 2024-09-12T02:22:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.639+0000 7f44deff5640 10 client.25876 handle_command_reply: tid=0 2024-09-12T02:22:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.639+0000 7f4504468740 1 client.25876 shutdown 2024-09-12T02:22:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.641+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 msgr2=0x557c36ce0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.641+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44f00bbae0 tx=0x7f44f00bb800 comp rx=0 tx=0).stop 2024-09-12T02:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 shutdown_connections 2024-09-12T02:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36cfd970 0x557c36cffdb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 --2- 172.21.15.3:0/3853907360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36ce0c70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 >> 172.21.15.3:0/3853907360 conn(0x557c36cd2010 msgr2=0x557c36cd3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f44deff5640 0 client.25876 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 shutdown_connections 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 1 -- 172.21.15.3:0/3853907360 wait complete. 2024-09-12T02:22:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.648+0000 7f4504468740 20 client.25876 trim_cache size 0 max 16384 2024-09-12T02:22:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 Processor -- start 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 -- start start 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36cd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800e5b0 con 0x557c36cdc370 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8010120 con 0x557c36cd72e0 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f4504468740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800ec80 con 0x557c36cd69a0 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36cd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55608/0 (socket says 172.21.15.3:55608) 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44faffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36cd09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49464/0 (socket says 172.21.15.3:49464) 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 -- 172.21.15.3:0/4094299313 learned_addr learned my addr 172.21.15.3:0/4094299313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 -- 172.21.15.3:0/4094299313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 msgr2=0x557c36cd09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36cd09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 -- 172.21.15.3:0/4094299313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 msgr2=0x557c36ce81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.663+0000 7f44fb7fe640 1 -- 172.21.15.3:0/4094299313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c36ce07b0 con 0x557c36cdc370 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.664+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f44f400ec90 tx=0x7f44f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e27710966737bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.664+0000 7f44e0ff9640 1 -- 172.21.15.3:0/4094299313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4025430 con 0x557c36cdc370 2024-09-12T02:22:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.664+0000 7f44e0ff9640 1 -- 172.21.15.3:0/4094299313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f401c900 con 0x557c36cdc370 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.664+0000 7f44e0ff9640 1 -- 172.21.15.3:0/4094299313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f401b7e0 con 0x557c36cdc370 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- 172.21.15.3:0/4094299313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 msgr2=0x557c36cd0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f44f400ec90 tx=0x7f44f4011000 comp rx=0 tx=0).stop 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- 172.21.15.3:0/4094299313 shutdown_connections 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36cd0ee0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd72e0 0x557c36ce81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- 172.21.15.3:0/4094299313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd69a0 0x557c36cd09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- 172.21.15.3:0/4094299313 >> 172.21.15.3:0/4094299313 conn(0x557c36cd2010 msgr2=0x557c36cd8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- 172.21.15.3:0/4094299313 shutdown_connections 2024-09-12T02:22:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- 172.21.15.3:0/4094299313 wait complete. 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- start start 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 Processor -- start 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36ce5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 0x557c36ce57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800fa20 con 0x557c36cdc370 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800f350 con 0x557c36cd69a0 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f44e800c590 con 0x557c36cd72e0 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f4504468740 10 client.0 _ll_register_callbacks cb 0x7f44f00bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f44fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.665+0000 7f44fa7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 0x557c36ce57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44faffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36ce5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55618/0 (socket says 172.21.15.3:55618) 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fb7fe640 1 -- 172.21.15.3:0/1159312403 learned_addr learned my addr 172.21.15.3:0/1159312403 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 -- 172.21.15.3:0/1159312403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 msgr2=0x557c36ce5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36ce5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 -- 172.21.15.3:0/1159312403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 msgr2=0x557c36ce67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 -- 172.21.15.3:0/1159312403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c36ce6dc0 con 0x557c36cd72e0 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fb7fe640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44faffd640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36ce5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:22:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44fa7fc640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 0x557c36ce57a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d40 tx=0x7f44ec003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f47f138bb50377 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44deff5640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec012430 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f401c900 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f44ec0125d0 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44ec0127d0 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f4504468740 10 client.25882 fetch_fsmap learned FSMap version 36 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.666+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c36ce8130 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f401c900 con 0x557c36cd72e0 2024-09-12T02:22:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 36) ==== 8366+0+0 (secure 0 0 0) 0x7f44ec011070 con 0x557c36cd72e0 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 10 client.25882 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 20 client.25882 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36d68710 0x557c36d6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 4 client.25882 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c36c41d70 con 0x557c36d68710 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f44faffd640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36d68710 0x557c36d6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f44faffd640 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36d68710 0x557c36d6ab00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbb4d1c249b58f2 server_cookie=fcacb42f6a24398c in_seq=0 out_seq=0 2024-09-12T02:22:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:07.667+0000 7f44deff5640 10 client.25882 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.111+0000 7f44deff5640 1 -- 172.21.15.3:0/1159312403 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c36c41d70 con 0x557c36d68710 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.111+0000 7f44deff5640 10 client.25882 handle_command_reply: tid=0 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.111+0000 7f4504468740 1 client.25882 shutdown 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.111+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 msgr2=0x557c36ce57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.111+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 0x557c36ce57a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009d40 tx=0x7f44ec003b30 comp rx=0 tx=0).stop 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 shutdown_connections 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x557c36d68710 0x557c36d6ab00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f44deff5640 0 client.25882 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557c36cdc370 0x557c36ce67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557c36cd72e0 0x557c36ce57a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 --2- 172.21.15.3:0/1159312403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557c36cd69a0 0x557c36ce5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 >> 172.21.15.3:0/1159312403 conn(0x557c36cd2010 msgr2=0x557c36cd2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 shutdown_connections 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.112+0000 7f4504468740 1 -- 172.21.15.3:0/1159312403 wait complete. 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.113+0000 7f4504468740 20 client.25882 trim_cache size 0 max 16384 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:22:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:22:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:22:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:22:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.113+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 msgr2=0x7f44e40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 0x7f44e40a5250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f44f40058c0 tx=0x7f44f4005850 comp rx=0 tx=0).stop 2024-09-12T02:22:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 msgr2=0x7f44fc071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44f0031610 tx=0x7f44f0032c80 comp rx=0 tx=0).stop 2024-09-12T02:22:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 shutdown_connections 2024-09-12T02:22:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f44e40a2d90 0x7f44e40a5250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f44fc1025f0 0x7f44fc073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44fc101c30 0x7f44fc0720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 --2- 172.21.15.3:0/2544412962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44fc100a40 0x7f44fc071bb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.114+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 >> 172.21.15.3:0/2544412962 conn(0x7f44fc0fc1c0 msgr2=0x7f44fc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.115+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 shutdown_connections 2024-09-12T02:22:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:08.115+0000 7f45015f8640 1 -- 172.21.15.3:0/2544412962 wait complete. 2024-09-12T02:22:08.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:07 smithi026 ceph-mon[35496]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 118 MiB/s wr, 18.80k op/s 2024-09-12T02:22:08.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:07 smithi003 ceph-mon[31476]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 118 MiB/s wr, 18.80k op/s 2024-09-12T02:22:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:07 smithi092 ceph-mon[36089]: pgmap v448: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 118 MiB/s wr, 18.80k op/s 2024-09-12T02:22:10.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:09 smithi026 ceph-mon[35496]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 116 MiB/s wr, 17.85k op/s 2024-09-12T02:22:10.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:09 smithi003 ceph-mon[31476]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 116 MiB/s wr, 17.85k op/s 2024-09-12T02:22:10.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:09 smithi092 ceph-mon[36089]: pgmap v449: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 116 MiB/s wr, 17.85k op/s 2024-09-12T02:22:12.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:11 smithi026 ceph-mon[35496]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 110 MiB/s wr, 16.94k op/s 2024-09-12T02:22:12.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:11 smithi003 ceph-mon[31476]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 110 MiB/s wr, 16.94k op/s 2024-09-12T02:22:12.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:11 smithi092 ceph-mon[36089]: pgmap v450: 97 pgs: 97 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 110 MiB/s wr, 16.94k op/s 2024-09-12T02:22:14.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:13 smithi026 ceph-mon[35496]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 112 MiB/s wr, 17.42k op/s 2024-09-12T02:22:14.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:13 smithi003 ceph-mon[31476]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 112 MiB/s wr, 17.42k op/s 2024-09-12T02:22:14.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:13 smithi092 ceph-mon[36089]: pgmap v451: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 112 MiB/s wr, 17.42k op/s 2024-09-12T02:22:16.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:15 smithi026 ceph-mon[35496]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-09-12T02:22:16.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:15 smithi003 ceph-mon[31476]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-09-12T02:22:16.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:15 smithi092 ceph-mon[36089]: pgmap v452: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 96 MiB/s wr, 14.97k op/s 2024-09-12T02:22:16.548 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T02:22:16.549 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:22:17.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:16 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:22:17.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:16 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:22:17.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:16 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:22:18.080 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:22:18.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:17 smithi026 ceph-mon[35496]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 MiB/s wr, 16.86k op/s 2024-09-12T02:22:18.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:17 smithi003 ceph-mon[31476]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 MiB/s wr, 16.86k op/s 2024-09-12T02:22:18.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:17 smithi092 ceph-mon[36089]: pgmap v453: 97 pgs: 97 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 109 MiB/s wr, 16.86k op/s 2024-09-12T02:22:20.011 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:22:20.052 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:22:20.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:19 smithi026 ceph-mon[35496]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 105 MiB/s wr, 16.27k op/s 2024-09-12T02:22:20.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:19 smithi003 ceph-mon[31476]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 105 MiB/s wr, 16.27k op/s 2024-09-12T02:22:20.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:19 smithi092 ceph-mon[36089]: pgmap v454: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 105 MiB/s wr, 16.27k op/s 2024-09-12T02:22:22.247 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:21 smithi003 ceph-mon[31476]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 86 MiB/s wr, 13.38k op/s 2024-09-12T02:22:22.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:21 smithi026 ceph-mon[35496]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 86 MiB/s wr, 13.38k op/s 2024-09-12T02:22:22.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:21 smithi092 ceph-mon[36089]: pgmap v455: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 86 MiB/s wr, 13.38k op/s 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.161+0000 7f5de8de8640 1 -- 172.21.15.3:0/3330135288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de410e630 msgr2=0x7f5de4110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.161+0000 7f5de8de8640 1 --2- 172.21.15.3:0/3330135288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de410e630 0x7f5de4110af0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00adb0 tx=0x7f5ddc02b7a0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.163+0000 7f5de8de8640 1 -- 172.21.15.3:0/3330135288 shutdown_connections 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.163+0000 7f5de8de8640 1 --2- 172.21.15.3:0/3330135288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de410e630 0x7f5de4110af0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.163+0000 7f5de8de8640 1 --2- 172.21.15.3:0/3330135288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.163+0000 7f5de8de8640 1 --2- 172.21.15.3:0/3330135288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de41091f0 0x7f5de410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.163+0000 7f5de8de8640 1 -- 172.21.15.3:0/3330135288 >> 172.21.15.3:0/3330135288 conn(0x7f5de406ecd0 msgr2=0x7f5de4071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:23.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.172+0000 7f5de8de8640 1 -- 172.21.15.3:0/3330135288 shutdown_connections 2024-09-12T02:22:23.175 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.175+0000 7f5de8de8640 1 -- 172.21.15.3:0/3330135288 wait complete. 2024-09-12T02:22:23.181 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.180+0000 7f5de8de8640 1 Processor -- start 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 -- start start 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de41091f0 0x7f5de419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de410e630 0x7f5de41b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4113dc0 con 0x7f5de41091f0 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4113c40 con 0x7f5de410e630 2024-09-12T02:22:23.182 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.181+0000 7f5de8de8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4113f40 con 0x7f5de410bb20 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:48306/0 (socket says 172.21.15.3:48306) 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 -- 172.21.15.3:0/4097849638 learned_addr learned my addr 172.21.15.3:0/4097849638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 -- 172.21.15.3:0/4097849638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de410e630 msgr2=0x7f5de41b6cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de410e630 0x7f5de41b6cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 -- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de41091f0 msgr2=0x7f5de419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de41091f0 0x7f5de419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.182+0000 7f5de1d9b640 1 -- 172.21.15.3:0/4097849638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de41b7200 con 0x7f5de410bb20 2024-09-12T02:22:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.183+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d770 tx=0x7f5dd800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.193+0000 7f5dd37fe640 1 -- 172.21.15.3:0/4097849638 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8017070 con 0x7f5de410bb20 2024-09-12T02:22:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.193+0000 7f5dd37fe640 1 -- 172.21.15.3:0/4097849638 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd8009c30 con 0x7f5de410bb20 2024-09-12T02:22:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.194+0000 7f5dd37fe640 1 -- 172.21.15.3:0/4097849638 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800f6b0 con 0x7f5de410bb20 2024-09-12T02:22:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.194+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de41b73d0 con 0x7f5de410bb20 2024-09-12T02:22:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.194+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41b7830 con 0x7f5de410bb20 2024-09-12T02:22:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.200+0000 7f5dd37fe640 1 -- 172.21.15.3:0/4097849638 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5dd801a070 con 0x7f5de410bb20 2024-09-12T02:22:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.201+0000 7f5dd37fe640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 0x7f5dcc0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.203+0000 7f5dd37fe640 1 -- 172.21.15.3:0/4097849638 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5dd8098ec0 con 0x7f5de410bb20 2024-09-12T02:22:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.207+0000 7f5de259c640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 0x7f5dcc0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.209 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.208+0000 7f5debc61740 1 Processor -- start 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.210+0000 7f5debc61740 1 -- start start 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 0x5620e6c3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 0x5620e6c41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 0x5620e6c44830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5620e6ba02f0 con 0x5620e6c3d2e0 2024-09-12T02:22:23.212 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620e6b48270 con 0x5620e6c42370 2024-09-12T02:22:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5debc61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5620e6b9c2b0 con 0x5620e6c3c9a0 2024-09-12T02:22:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5de2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 0x5620e6c44830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5de2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 0x5620e6c44830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:37604/0 (socket says 172.21.15.3:37604) 2024-09-12T02:22:23.213 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.211+0000 7f5de2d9d640 1 -- 172.21.15.3:0/305895281 learned_addr learned my addr 172.21.15.3:0/305895281 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.214+0000 7f5de259c640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 0x5620e6c3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.214+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 0x5620e6c41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 -- 172.21.15.3:0/305895281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 msgr2=0x5620e6c44830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 0x5620e6c44830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 -- 172.21.15.3:0/305895281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 msgr2=0x5620e6c41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 0x5620e6c41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 -- 172.21.15.3:0/305895281 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e6b47340 con 0x5620e6c3c9a0 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 0x5620e6c41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:22:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.215+0000 7f5de259c640 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 0x5620e6c3cda0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40095f0 tx=0x7f5dd4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feeff1be7a32fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.216+0000 7f5de259c640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 0x7f5dcc0a4fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005d00 tx=0x7f5dd4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.216+0000 7f5dd0ff9640 1 -- 172.21.15.3:0/305895281 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4010d90 con 0x5620e6c3c9a0 2024-09-12T02:22:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.216+0000 7f5dd0ff9640 1 -- 172.21.15.3:0/305895281 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd4005500 con 0x5620e6c3c9a0 2024-09-12T02:22:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.217+0000 7f5debc61740 1 -- 172.21.15.3:0/305895281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 msgr2=0x5620e6c3cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.217+0000 7f5debc61740 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 0x5620e6c3cda0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40095f0 tx=0x7f5dd4019000 comp rx=0 tx=0).stop 2024-09-12T02:22:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 -- 172.21.15.3:0/305895281 shutdown_connections 2024-09-12T02:22:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c42370 0x5620e6c44830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3d2e0 0x5620e6c41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 --2- 172.21.15.3:0/305895281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3c9a0 0x5620e6c3cda0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 -- 172.21.15.3:0/305895281 >> 172.21.15.3:0/305895281 conn(0x5620e6c38010 msgr2=0x5620e6c38410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.221+0000 7f5debc61740 1 -- 172.21.15.3:0/305895281 shutdown_connections 2024-09-12T02:22:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.223+0000 7f5debc61740 1 -- 172.21.15.3:0/305895281 wait complete. 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.224+0000 7f5debc61740 1 -- start start 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.224+0000 7f5debc61740 1 Processor -- start 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c47140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5620e6c45390 con 0x5620e6c3c9a0 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5620e6b29290 con 0x5620e6c3d2e0 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5620e6c45510 con 0x5620e6c42370 2024-09-12T02:22:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5debc61740 10 client.0 _ll_register_callbacks cb 0x5620e6b5a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5de259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5de259c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36600/0 (socket says 172.21.15.3:36600) 2024-09-12T02:22:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5de259c640 1 -- 172.21.15.3:0/1722049967 learned_addr learned my addr 172.21.15.3:0/1722049967 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.225+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c47140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 -- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 msgr2=0x5620e6c4e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 -- 172.21.15.3:0/1722049967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 msgr2=0x5620e6c47140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c47140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 -- 172.21.15.3:0/1722049967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e68b3210 con 0x5620e6c3c9a0 2024-09-12T02:22:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.226+0000 7f5de259c640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e720 tx=0x7f5dd4004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a388a57ab23039d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.227+0000 7f5dc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.227+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40047a0 con 0x5620e6c3c9a0 2024-09-12T02:22:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.227+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd4005500 con 0x5620e6c3c9a0 2024-09-12T02:22:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.227+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5dd4004f40 con 0x5620e6c3c9a0 2024-09-12T02:22:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.228+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5dd4005140 con 0x5620e6c3c9a0 2024-09-12T02:22:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.228+0000 7f5debc61740 10 client.16302 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.228+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5620e697f620 con 0x5620e6c3c9a0 2024-09-12T02:22:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd4005500 con 0x5620e6c3c9a0 2024-09-12T02:22:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f5dd401b800 con 0x5620e6c3c9a0 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 10 client.16302 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 20 client.16302 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6c52550 0x5620e6c54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 4 client.16302 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.229+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620e6bada20 con 0x5620e6c52550 2024-09-12T02:22:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.234+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6c52550 0x5620e6c54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.236+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6c52550 0x5620e6c54940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550c6946ef77215c server_cookie=68477831372b845f in_seq=0 out_seq=0 2024-09-12T02:22:23.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.236+0000 7f5dc9ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/1722049967 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620e6bada20 con 0x5620e6c52550 2024-09-12T02:22:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5dc9ffb640 10 client.16302 handle_command_reply: tid=0 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 client.16302 shutdown 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 msgr2=0x5620e6c46c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400e720 tx=0x7f5dd4004b60 comp rx=0 tx=0).stop 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 shutdown_connections 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6c52550 0x5620e6c54940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c47140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 --2- 172.21.15.3:0/1722049967 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c46c00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.239+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 >> 172.21.15.3:0/1722049967 conn(0x5620e6c38010 msgr2=0x5620e6c39820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.241+0000 7f5dc9ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.241+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 shutdown_connections 2024-09-12T02:22:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.241+0000 7f5debc61740 1 -- 172.21.15.3:0/1722049967 wait complete. 2024-09-12T02:22:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.241+0000 7f5debc61740 20 client.16302 trim_cache size 0 max 16384 2024-09-12T02:22:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.302+0000 7f5debc61740 1 Processor -- start 2024-09-12T02:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5debc61740 1 -- start start 2024-09-12T02:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5debc61740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 0x5620e6c38ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5debc61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5debc61740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5debc61740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41c0670 con 0x5620e6c42370 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5de2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5de2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36614/0 (socket says 172.21.15.3:36614) 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.308+0000 7f5de2d9d640 1 -- 172.21.15.3:0/1373897101 learned_addr learned my addr 172.21.15.3:0/1373897101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.309+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41c21b0 con 0x5620e6c3c9a0 2024-09-12T02:22:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.309+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41c0d40 con 0x5620e6c3d2e0 2024-09-12T02:22:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.309+0000 7f5de259c640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 0x5620e6c38ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.309+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.312+0000 7f5de2d9d640 1 -- 172.21.15.3:0/1373897101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 msgr2=0x5620e6c4e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.314+0000 7f5de2d9d640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.314+0000 7f5de2d9d640 1 -- 172.21.15.3:0/1373897101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 msgr2=0x5620e6c38ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.314+0000 7f5de2d9d640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 0x5620e6c38ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.314+0000 7f5de2d9d640 1 -- 172.21.15.3:0/1373897101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e6c46740 con 0x5620e6c42370 2024-09-12T02:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.314+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.315+0000 7f5de259c640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 0x5620e6c38ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.316+0000 7f5de2d9d640 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00adb0 tx=0x7f5ddc00ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4050a9fcde4ea10e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.317+0000 7f5dcbfff640 1 -- 172.21.15.3:0/1373897101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc030710 con 0x5620e6c42370 2024-09-12T02:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.317+0000 7f5dcbfff640 1 -- 172.21.15.3:0/1373897101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc003e50 con 0x5620e6c42370 2024-09-12T02:22:23.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.317+0000 7f5dcbfff640 1 -- 172.21.15.3:0/1373897101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc032660 con 0x5620e6c42370 2024-09-12T02:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 msgr2=0x5620e6c39400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00adb0 tx=0x7f5ddc00ab30 comp rx=0 tx=0).stop 2024-09-12T02:22:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 shutdown_connections 2024-09-12T02:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c42370 0x5620e6c39400 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 --2- 172.21.15.3:0/1373897101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3c9a0 0x5620e6c38ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.319+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 >> 172.21.15.3:0/1373897101 conn(0x5620e6c38010 msgr2=0x5620e6c46bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.322+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 shutdown_connections 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.325+0000 7f5debc61740 1 -- 172.21.15.3:0/1373897101 wait complete. 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.325+0000 7f5debc61740 1 -- start start 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 Processor -- start 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41c1ae0 con 0x5620e6c3c9a0 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41c1410 con 0x5620e6c3d2e0 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.326+0000 7f5debc61740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41be650 con 0x5620e6c42370 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:37626/0 (socket says 172.21.15.3:37626) 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 -- 172.21.15.3:0/327956112 learned_addr learned my addr 172.21.15.3:0/327956112 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5debc61740 10 client.0 _ll_register_callbacks cb 0x7f5dd4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de2d9d640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 -- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 msgr2=0x5620e6c4cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 -- 172.21.15.3:0/327956112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 msgr2=0x5620e6c4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.327+0000 7f5de1d9b640 1 -- 172.21.15.3:0/327956112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e6c4d270 con 0x5620e6c3d2e0 2024-09-12T02:22:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.329+0000 7f5de1d9b640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8010db0 tx=0x7f5dd809a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00d8cfd73138d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.329+0000 7f5dc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:22:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.330+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80a7020 con 0x5620e6c3d2e0 2024-09-12T02:22:23.330 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.331+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5dd809a470 con 0x5620e6c3d2e0 2024-09-12T02:22:23.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.332+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc003e50 con 0x5620e6c3d2e0 2024-09-12T02:22:23.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.335+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80cc210 con 0x5620e6c3d2e0 2024-09-12T02:22:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.336+0000 7f5debc61740 10 client.25760 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.336+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5620e6cbd1c0 con 0x5620e6c3d2e0 2024-09-12T02:22:23.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.344+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc003e50 con 0x5620e6c3d2e0 2024-09-12T02:22:23.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.347+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f5dd80a3ce0 con 0x5620e6c3d2e0 2024-09-12T02:22:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 10 client.25760 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 20 client.25760 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6cbd930 0x5620e6cbfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 4 client.25760 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:22:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.348+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620e6ba7d70 con 0x5620e6cbd930 2024-09-12T02:22:23.352 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.352+0000 7f5de259c640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6cbd930 0x5620e6cbfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.353+0000 7f5de259c640 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6cbd930 0x5620e6cbfd50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d808146514c90247 server_cookie=b4f33ac7c5661c9d in_seq=0 out_seq=0 2024-09-12T02:22:23.353 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:23.353+0000 7f5dc9ffb640 10 client.25760 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.019+0000 7f5dc9ffb640 1 -- 172.21.15.3:0/327956112 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620e6ba7d70 con 0x5620e6cbd930 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5dc9ffb640 10 client.25760 handle_command_reply: tid=0 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 client.25760 shutdown 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 msgr2=0x5620e6c4bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8010db0 tx=0x7f5dd809a9e0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 shutdown_connections 2024-09-12T02:22:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5620e6cbd930 0x5620e6cbfd50 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5620e6c42370 0x5620e6c4cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5620e6c3d2e0 0x5620e6c4bca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 --2- 172.21.15.3:0/327956112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5620e6c3c9a0 0x5620e6c4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 >> 172.21.15.3:0/327956112 conn(0x5620e6c38010 msgr2=0x5620e6c369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.020+0000 7f5dc9ffb640 0 client.25760 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.021+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 shutdown_connections 2024-09-12T02:22:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.021+0000 7f5debc61740 1 -- 172.21.15.3:0/327956112 wait complete. 2024-09-12T02:22:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.021+0000 7f5debc61740 20 client.25760 trim_cache size 0 max 16384 2024-09-12T02:22:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:22:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:22:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:22:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:22:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 msgr2=0x7f5dcc0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 0x7f5dcc0a4fe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4005d00 tx=0x7f5dd4005c50 comp rx=0 tx=0).stop 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 msgr2=0x7f5de419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800d770 tx=0x7f5dd800dc30 comp rx=0 tx=0).stop 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 shutdown_connections 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f5dcc0a2b20 0x7f5dcc0a4fe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de410e630 0x7f5de41b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5de410bb20 0x7f5de419e930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.022+0000 7f5de8de8640 1 --2- 172.21.15.3:0/4097849638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5de41091f0 0x7f5de419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.023+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 >> 172.21.15.3:0/4097849638 conn(0x7f5de406ecd0 msgr2=0x7f5de410a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.024+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 shutdown_connections 2024-09-12T02:22:24.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:24.024+0000 7f5de8de8640 1 -- 172.21.15.3:0/4097849638 wait complete. 2024-09-12T02:22:24.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:23 smithi026 ceph-mon[35496]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 97 MiB/s wr, 15.11k op/s 2024-09-12T02:22:24.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:23 smithi003 ceph-mon[31476]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 97 MiB/s wr, 15.11k op/s 2024-09-12T02:22:24.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:23 smithi092 ceph-mon[36089]: pgmap v456: 97 pgs: 97 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 97 MiB/s wr, 15.11k op/s 2024-09-12T02:22:26.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:25 smithi026 ceph-mon[35496]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 99 MiB/s wr, 15.28k op/s 2024-09-12T02:22:26.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:25 smithi003 ceph-mon[31476]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 99 MiB/s wr, 15.28k op/s 2024-09-12T02:22:26.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:25 smithi092 ceph-mon[36089]: pgmap v457: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 99 MiB/s wr, 15.28k op/s 2024-09-12T02:22:28.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:27 smithi026 ceph-mon[35496]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 109 MiB/s wr, 16.51k op/s 2024-09-12T02:22:28.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:27 smithi003 ceph-mon[31476]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 109 MiB/s wr, 16.51k op/s 2024-09-12T02:22:28.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:27 smithi092 ceph-mon[36089]: pgmap v458: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 109 MiB/s wr, 16.51k op/s 2024-09-12T02:22:29.993 INFO:teuthology.orchestra.run.smithi003.stdout:[] 2024-09-12T02:22:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 -- 172.21.15.3:0/1608714302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c806ea30 msgr2=0x7f88c806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:29.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 --2- 172.21.15.3:0/1608714302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c806ea30 0x7f88c806ee90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f88b801a910 tx=0x7f88b803b470 comp rx=0 tx=0).stop 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 -- 172.21.15.3:0/1608714302 shutdown_connections 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 --2- 172.21.15.3:0/1608714302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c811b140 0x7f88c811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 --2- 172.21.15.3:0/1608714302 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c806ea30 0x7f88c806ee90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 --2- 172.21.15.3:0/1608714302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88c8110220 0x7f88c806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.675+0000 7f88cff4e640 1 -- 172.21.15.3:0/1608714302 >> 172.21.15.3:0/1608714302 conn(0x7f88c8073210 msgr2=0x7f88c8073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.676+0000 7f88cff4e640 1 -- 172.21.15.3:0/1608714302 shutdown_connections 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.676+0000 7f88cff4e640 1 -- 172.21.15.3:0/1608714302 wait complete. 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 Processor -- start 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 -- start start 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c806ea30 0x7f88c811a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88c811b140 0x7f88c8116e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88c81133e0 con 0x7f88c8110220 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8113260 con 0x7f88c811b140 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.677+0000 7f88cff4e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88c8113560 con 0x7f88c806ea30 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.678+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.678+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60338/0 (socket says 172.21.15.3:60338) 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.678+0000 7f88cd4e9640 1 -- 172.21.15.3:0/4136944219 learned_addr learned my addr 172.21.15.3:0/4136944219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.679+0000 7f88cd4e9640 1 -- 172.21.15.3:0/4136944219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c806ea30 msgr2=0x7f88c811a870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.679+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c806ea30 0x7f88c811a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.679+0000 7f88cd4e9640 1 -- 172.21.15.3:0/4136944219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88c811b140 msgr2=0x7f88c8116e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.679+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88c811b140 0x7f88c8116e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.679+0000 7f88cd4e9640 1 -- 172.21.15.3:0/4136944219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c8117580 con 0x7f88c8110220 2024-09-12T02:22:29.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.680+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f88b801aa30 tx=0x7f88b8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.682+0000 7f88beffd640 1 -- 172.21.15.3:0/4136944219 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b804f070 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.682+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81ab0a0 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.682+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81ab440 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.682+0000 7f88beffd640 1 -- 172.21.15.3:0/4136944219 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b8040c00 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.682+0000 7f88beffd640 1 -- 172.21.15.3:0/4136944219 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8046030 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.686+0000 7f88beffd640 1 -- 172.21.15.3:0/4136944219 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f88b8052070 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.688+0000 7f88beffd640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 0x7f88b40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.688+0000 7f88beffd640 1 -- 172.21.15.3:0/4136944219 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f88b8100480 con 0x7f88c8110220 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.693+0000 7f88d2327740 1 Processor -- start 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.693+0000 7f88d2327740 1 -- start start 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303ccf0 0x56050303b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560503046520 0x560503048910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560502fa12f0 con 0x560503046520 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560502f48cf0 con 0x56050303b780 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.694+0000 7f88d2327740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560502f9d2b0 con 0x56050303ccf0 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.695+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.695+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:42378/0 (socket says 172.21.15.3:42378) 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.695+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2088295556 learned_addr learned my addr 172.21.15.3:0/2088295556 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.696+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2088295556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303ccf0 msgr2=0x56050303b240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.696+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303ccf0 0x56050303b240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.696+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2088295556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560503046520 msgr2=0x560503048910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.696+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560503046520 0x560503048910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.696+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2088295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560502f2d3a0 con 0x56050303b780 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.697+0000 7f88cdcea640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 0x7f88b40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.697+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88b80d5410 tx=0x7f88b803b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d935f3c065576670 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.698+0000 7f88b3fff640 1 -- 172.21.15.3:0/2088295556 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8059040 con 0x56050303b780 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.698+0000 7f88b3fff640 1 -- 172.21.15.3:0/2088295556 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80cafd0 con 0x56050303b780 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.698+0000 7f88b3fff640 1 -- 172.21.15.3:0/2088295556 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b81000e0 con 0x56050303b780 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.702+0000 7f88d2327740 1 -- 172.21.15.3:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 msgr2=0x56050303bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.702+0000 7f88d2327740 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88b80d5410 tx=0x7f88b803b470 comp rx=0 tx=0).stop 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.703+0000 7f88d2327740 1 -- 172.21.15.3:0/2088295556 shutdown_connections 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.703+0000 7f88d2327740 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560503046520 0x560503048910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.703+0000 7f88d2327740 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x56050303bbc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:29.999 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.703+0000 7f88d2327740 1 --2- 172.21.15.3:0/2088295556 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303ccf0 0x56050303b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.703+0000 7f88d2327740 1 -- 172.21.15.3:0/2088295556 >> 172.21.15.3:0/2088295556 conn(0x560503004310 msgr2=0x560503037e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.709+0000 7f88d2327740 1 -- 172.21.15.3:0/2088295556 shutdown_connections 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.712+0000 7f88d2327740 1 -- 172.21.15.3:0/2088295556 wait complete. 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.713+0000 7f88d2327740 1 -- start start 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.713+0000 7f88d2327740 1 Processor -- start 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88cdcea640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 0x7f88b40a51d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88c000abd0 tx=0x7f88c0009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 0x560503045f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 0x5605030431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x5605030437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560503048fe0 con 0x56050303b780 2024-09-12T02:22:30.000 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560502f5c060 con 0x56050303ccf0 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560503049160 con 0x560503046520 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.714+0000 7f88d2327740 10 client.0 _ll_register_callbacks cb 0x560502caf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cdcea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 0x560503045f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cdcea640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 0x560503045f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60350/0 (socket says 172.21.15.3:60350) 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cdcea640 1 -- 172.21.15.3:0/2401267918 learned_addr learned my addr 172.21.15.3:0/2401267918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 0x5605030431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88ce4eb640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x5605030437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 msgr2=0x5605030437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x5605030437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2401267918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 msgr2=0x560503045f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 0x560503045f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.715+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2401267918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560502d123c0 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.716+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 0x5605030431c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560503038c70 tx=0x7f88b80057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ed106331ed218c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.717+0000 7f88b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.717+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80e9030 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.718+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b80cafd0 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.718+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f88b81000f0 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.718+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f88b8059040 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.719+0000 7f88d2327740 10 client.25766 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.719+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560503048fe0 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.719+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80cafd0 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.719+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f88b80e4020 con 0x56050303ccf0 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 10 client.25766 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:30.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 20 client.25766 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560503055e90 0x5605030582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 4 client.25766 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.720+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560502fae860 con 0x560503055e90 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.721+0000 7f88cdcea640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560503055e90 0x5605030582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.731+0000 7f88cdcea640 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560503055e90 0x5605030582b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682b7bef5cfed32e server_cookie=de3bfb16eed6ce18 in_seq=0 out_seq=0 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.731+0000 7f88b17fa640 10 client.25766 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.733+0000 7f88b17fa640 1 -- 172.21.15.3:0/2401267918 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560502fae860 con 0x560503055e90 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.733+0000 7f88b17fa640 10 client.25766 handle_command_reply: tid=0 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.733+0000 7f88d2327740 1 client.25766 shutdown 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.733+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 msgr2=0x5605030431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.733+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 0x5605030431c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560503038c70 tx=0x7f88b80057e0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 shutdown_connections 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560503055e90 0x5605030582b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x5605030437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303ccf0 0x5605030431c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 --2- 172.21.15.3:0/2401267918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303b780 0x560503045f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 >> 172.21.15.3:0/2401267918 conn(0x560503004310 msgr2=0x56050303db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88b17fa640 0 client.25766 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 shutdown_connections 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.734+0000 7f88d2327740 1 -- 172.21.15.3:0/2401267918 wait complete. 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.735+0000 7f88d2327740 20 client.25766 trim_cache size 0 max 16384 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.764+0000 7f88d2327740 1 Processor -- start 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.766+0000 7f88d2327740 1 -- start start 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88d2327740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x560503052240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88d2327740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503042a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88cdcea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x560503052240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88cdcea640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x560503052240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:42388/0 (socket says 172.21.15.3:42388) 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88cdcea640 1 -- 172.21.15.3:0/3299348592 learned_addr learned my addr 172.21.15.3:0/3299348592 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.767+0000 7f88d2327740 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x56050305db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.768+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac00e570 con 0x56050303ccf0 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.768+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac0100b0 con 0x56050303b780 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.768+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac00ec40 con 0x560503046520 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.768+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503042a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.768+0000 7f88ce4eb640 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x56050305db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 -- 172.21.15.3:0/3299348592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 msgr2=0x56050305db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x56050305db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 -- 172.21.15.3:0/3299348592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 msgr2=0x560503052240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x560503052240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 -- 172.21.15.3:0/3299348592 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560502fb4e70 con 0x56050303ccf0 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.769+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503042a80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x560503044770 tx=0x7f88b80cce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7388dee86fa32776 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.773+0000 7f88b37fe640 1 -- 172.21.15.3:0/3299348592 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80cc260 con 0x56050303ccf0 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.773+0000 7f88b37fe640 1 -- 172.21.15.3:0/3299348592 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80cc400 con 0x56050303ccf0 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.773+0000 7f88b37fe640 1 -- 172.21.15.3:0/3299348592 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8042bb0 con 0x56050303ccf0 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 msgr2=0x560503042a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503042a80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x560503044770 tx=0x7f88b80cce50 comp rx=0 tx=0).stop 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 shutdown_connections 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560503046520 0x56050305db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503042a80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.774+0000 7f88d2327740 1 --2- 172.21.15.3:0/3299348592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56050303b780 0x560503052240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.775+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 >> 172.21.15.3:0/3299348592 conn(0x560503004310 msgr2=0x56050303e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.780+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 shutdown_connections 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.782+0000 7f88d2327740 1 -- 172.21.15.3:0/3299348592 wait complete. 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.782+0000 7f88d2327740 1 -- start start 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.782+0000 7f88d2327740 1 Processor -- start 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88d2327740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303b780 0x56050304ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88d2327740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88d2327740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560503046520 0x5605030498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88d2327740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac00f9e0 con 0x56050303ccf0 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88d2327740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac00f310 con 0x560503046520 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88cd4e9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60376/0 (socket says 172.21.15.3:60376) 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.783+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2868685981 learned_addr learned my addr 172.21.15.3:0/2868685981 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:30.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.784+0000 7f88cdcea640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303b780 0x56050304ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.784+0000 7f88ce4eb640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560503046520 0x5605030498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.784+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac00c550 con 0x56050303b780 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.784+0000 7f88d2327740 10 client.0 _ll_register_callbacks cb 0x7f88b80cd3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.787+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303b780 msgr2=0x56050304ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.787+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303b780 0x56050304ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.787+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2868685981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560503046520 msgr2=0x5605030498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.787+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560503046520 0x5605030498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.787+0000 7f88cd4e9640 1 -- 172.21.15.3:0/2868685981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050304a0e0 con 0x56050303ccf0 2024-09-12T02:22:30.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.791+0000 7f88cd4e9640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88b80c9130 tx=0x7f88b803b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5790663cb410250d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.791+0000 7f88b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80e9070 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f88b80d56c0 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b80cc400 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f88b80cc260 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88d2327740 10 client.16320 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56050304a0e0 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.792+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b80abbc0 con 0x56050303ccf0 2024-09-12T02:22:30.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f88b80e4070 con 0x56050303ccf0 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 10 client.16320 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 20 client.16320 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5605030bfcf0 0x5605030c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 4 client.16320 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f88b80cc400 con 0x5605030bfcf0 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88cdcea640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5605030bfcf0 0x5605030c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88cdcea640 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5605030bfcf0 0x5605030c20e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51fc92dcf22ae594 server_cookie=567e8ce9c2b1d8f6 in_seq=0 out_seq=0 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.793+0000 7f88b17fa640 10 client.16320 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:30.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88b17fa640 1 -- 172.21.15.3:0/2868685981 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f88b80cc400 con 0x5605030bfcf0 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88b17fa640 10 client.16320 handle_command_reply: tid=0 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 client.16320 shutdown 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 msgr2=0x560503049370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f88b80c9130 tx=0x7f88b803b470 comp rx=0 tx=0).stop 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 shutdown_connections 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5605030bfcf0 0x5605030c20e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560503046520 0x5605030498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56050303ccf0 0x560503049370 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 --2- 172.21.15.3:0/2868685981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56050303b780 0x56050304ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 >> 172.21.15.3:0/2868685981 conn(0x560503004310 msgr2=0x560503036690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88b17fa640 0 client.16320 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 shutdown_connections 2024-09-12T02:22:30.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 1 -- 172.21.15.3:0/2868685981 wait complete. 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.794+0000 7f88d2327740 20 client.16320 trim_cache size 0 max 16384 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 msgr2=0x7f88b40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 0x7f88b40a51d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f88c000abd0 tx=0x7f88c0009250 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 msgr2=0x7f88c8116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f88b801aa30 tx=0x7f88b8004cf0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 shutdown_connections 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f88b40a2d10 0x7f88b40a51d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88c811b140 0x7f88c8116e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88c8110220 0x7f88c8116910 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 --2- 172.21.15.3:0/4136944219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f88c806ea30 0x7f88c811a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 >> 172.21.15.3:0/4136944219 conn(0x7f88c8073210 msgr2=0x7f88c811b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 shutdown_connections 2024-09-12T02:22:30.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:28.795+0000 7f88cff4e640 1 -- 172.21.15.3:0/4136944219 wait complete. 2024-09-12T02:22:30.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:29 smithi026 ceph-mon[35496]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 98 MiB/s wr, 14.87k op/s 2024-09-12T02:22:30.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:29 smithi003 ceph-mon[31476]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 98 MiB/s wr, 14.87k op/s 2024-09-12T02:22:30.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:29 smithi092 ceph-mon[36089]: pgmap v459: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 98 MiB/s wr, 14.87k op/s 2024-09-12T02:22:32.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:31 smithi026 ceph-mon[35496]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 95 MiB/s wr, 14.13k op/s 2024-09-12T02:22:32.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:31 smithi003 ceph-mon[31476]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 95 MiB/s wr, 14.13k op/s 2024-09-12T02:22:32.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:31 smithi092 ceph-mon[36089]: pgmap v460: 97 pgs: 97 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 95 MiB/s wr, 14.13k op/s 2024-09-12T02:22:34.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:33 smithi026 ceph-mon[35496]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 104 MiB/s wr, 15.34k op/s 2024-09-12T02:22:34.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:33 smithi003 ceph-mon[31476]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 104 MiB/s wr, 15.34k op/s 2024-09-12T02:22:34.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:33 smithi092 ceph-mon[36089]: pgmap v461: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 104 MiB/s wr, 15.34k op/s 2024-09-12T02:22:36.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:35 smithi026 ceph-mon[35496]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-09-12T02:22:36.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:35 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1661735370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:22:36.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:35 smithi003 ceph-mon[31476]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-09-12T02:22:36.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:35 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1661735370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:22:36.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:35 smithi092 ceph-mon[36089]: pgmap v462: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-09-12T02:22:36.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:35 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1661735370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:22:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.611+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3809983842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84105510 msgr2=0x7f9b841059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.611+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3809983842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84105510 0x7f9b841059c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00ac70 tx=0x7f9b7c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.617+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3809983842 shutdown_connections 2024-09-12T02:22:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.617+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3809983842 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84105510 0x7f9b841059c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.617+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3809983842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84102b60 0x7f9b84104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.617+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3809983842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84100210 0x7f9b84102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.617+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3809983842 >> 172.21.15.3:0/3809983842 conn(0x7f9b840fc000 msgr2=0x7f9b840fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.622+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3809983842 shutdown_connections 2024-09-12T02:22:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.624+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3809983842 wait complete. 2024-09-12T02:22:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.624+0000 7f9b8c34b640 1 Processor -- start 2024-09-12T02:22:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 -- start start 2024-09-12T02:22:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 0x7f9b84195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 0x7f9b84196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84105510 0x7f9b8419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84109470 con 0x7f9b84100210 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841092f0 con 0x7f9b84105510 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.627+0000 7f9b8c34b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841095f0 con 0x7f9b84102b60 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 0x7f9b84195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 0x7f9b84195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60410/0 (socket says 172.21.15.3:60410) 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/3108349426 learned_addr learned my addr 172.21.15.3:0/3108349426 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b8a8e8640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84105510 0x7f9b8419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 0x7f9b84196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 -- 172.21.15.3:0/3108349426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84105510 msgr2=0x7f9b8419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84105510 0x7f9b8419d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 -- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 msgr2=0x7f9b84195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 0x7f9b84195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.628+0000 7f9b898e6640 1 -- 172.21.15.3:0/3108349426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8419d700 con 0x7f9b84102b60 2024-09-12T02:22:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.629+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 0x7f9b84196020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b8001f9b0 tx=0x7f9b8001fe70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.630+0000 7f9b7b7fe640 1 -- 172.21.15.3:0/3108349426 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8001db90 con 0x7f9b84102b60 2024-09-12T02:22:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.630+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8419d9f0 con 0x7f9b84102b60 2024-09-12T02:22:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.630+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8419dfc0 con 0x7f9b84102b60 2024-09-12T02:22:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.630+0000 7f9b7b7fe640 1 -- 172.21.15.3:0/3108349426 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8001dd30 con 0x7f9b84102b60 2024-09-12T02:22:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.630+0000 7f9b7b7fe640 1 -- 172.21.15.3:0/3108349426 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80029840 con 0x7f9b84102b60 2024-09-12T02:22:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.636+0000 7f9b7b7fe640 1 -- 172.21.15.3:0/3108349426 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9b800299e0 con 0x7f9b84102b60 2024-09-12T02:22:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.638+0000 7f9b7b7fe640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 0x7f9b700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.638+0000 7f9b7b7fe640 1 -- 172.21.15.3:0/3108349426 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9b800e1310 con 0x7f9b84102b60 2024-09-12T02:22:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.639+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 0x7f9b700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:22:37.720 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":37,"btime":"2024-09-12T02:22:15:807901+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:22:15.541633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:22:37.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.137+0000 7f0003fff640 1 -- 172.21.15.3:0/2962770914 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c09d320 con 0x7f00040a2040 2024-09-12T02:22:37.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.137+0000 7f001216c640 1 -- 172.21.15.3:0/2962770914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00040a2040 msgr2=0x7f00040a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.137+0000 7f001216c640 1 --2- 172.21.15.3:0/2962770914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00040a2040 0x7f00040a4430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f000c067960 tx=0x7f000c096dd0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.138+0000 7f001216c640 1 -- 172.21.15.3:0/2962770914 shutdown_connections 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.138+0000 7f001216c640 1 --2- 172.21.15.3:0/2962770914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00040a2040 0x7f00040a4430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.138+0000 7f001216c640 1 --2- 172.21.15.3:0/2962770914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004006680 0x7f0004006ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.138+0000 7f001216c640 1 --2- 172.21.15.3:0/2962770914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0004096520 0x7f0004006140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.138+0000 7f001216c640 1 -- 172.21.15.3:0/2962770914 >> 172.21.15.3:0/2962770914 conn(0x7f000400a290 msgr2=0x7f000400a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.149+0000 7f001216c640 1 -- 172.21.15.3:0/2962770914 shutdown_connections 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.154+0000 7f001216c640 1 -- 172.21.15.3:0/2962770914 wait complete. 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.155+0000 7f001216c640 1 Processor -- start 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.155+0000 7f001216c640 1 -- start start 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.155+0000 7f001216c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 0x7f00040a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.155+0000 7f001216c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004096520 0x7f000409ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.155+0000 7f001216c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001216c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f00040a7ba0 con 0x7f00040a2040 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001216c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f00040a7a20 con 0x7f0004096520 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001216c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f00040a7d20 con 0x7f0004006680 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60392/0 (socket says 172.21.15.3:60392) 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 -- 172.21.15.3:0/1661735370 learned_addr learned my addr 172.21.15.3:0/1661735370 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001116a640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 0x7f00040a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f0010969640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004096520 0x7f000409ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 -- 172.21.15.3:0/1661735370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 msgr2=0x7f00040a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 0x7f00040a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 -- 172.21.15.3:0/1661735370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004096520 msgr2=0x7f000409ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004096520 0x7f000409ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001196b640 1 -- 172.21.15.3:0/1661735370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000409f800 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.156+0000 7f001116a640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 0x7f00040a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.157+0000 7f001196b640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f000c065a40 tx=0x7f000c09d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.158+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0ad020 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.158+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c09dcd0 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.159+0000 7f001216c640 1 -- 172.21.15.3:0/1661735370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000413e980 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.159+0000 7f001216c640 1 -- 172.21.15.3:0/1661735370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000413eee0 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.160+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c09f700 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.161+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0004094be0 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.164+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f000c0b0020 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.165+0000 7f00027fc640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 0x7efff80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.165+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f000c15d100 con 0x7f00040a2040 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.166+0000 7f001116a640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 0x7efff80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.167+0000 7f001116a640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 0x7efff80a50d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0008009f40 tx=0x7f0008005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.175+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f000c0b7020 con 0x7f00040a2040 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.452+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0004006ac0 con 0x7f00040a2040 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.455+0000 7f00027fc640 1 -- 172.21.15.3:0/1661735370 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11946 (secure 0 0 0) 0x7f000c12a050 con 0x7f00040a2040 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 37 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 msgr2=0x7efff80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 0x7efff80a50d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0008009f40 tx=0x7f0008005f90 comp rx=0 tx=0).stop 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 msgr2=0x7f000409ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f000c065a40 tx=0x7f000c09d610 comp rx=0 tx=0).stop 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 shutdown_connections 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efff80a2c10 0x7efff80a50d0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0008009f40 tx=0x7f0008005f90 comp rx=0 tx=0).stop 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f00040a2040 0x7f000409ef70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0004096520 0x7f000409ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 --2- 172.21.15.3:0/1661735370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0004006680 0x7f00040a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 >> 172.21.15.3:0/1661735370 conn(0x7f000400a290 msgr2=0x7f00040a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.460+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 shutdown_connections 2024-09-12T02:22:37.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:22:35.463+0000 7efff7fff640 1 -- 172.21.15.3:0/1661735370 wait complete. 2024-09-12T02:22:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.728+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 0x7f9b700a5260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009a30 tx=0x7f9b74009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.729+0000 7f9b8f1c4740 1 Processor -- start 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.729+0000 7f9b8f1c4740 1 -- start start 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c04b70 0x565194c04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c0ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565194b682f0 con 0x565194c054b0 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565194b10270 con 0x565194c0a540 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565194b642b0 con 0x565194c04b70 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8a8e8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c0ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60424/0 (socket says 172.21.15.3:60424) 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 -- 172.21.15.3:0/1085455508 learned_addr learned my addr 172.21.15.3:0/1085455508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c04b70 0x565194c04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 -- 172.21.15.3:0/1085455508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c04b70 msgr2=0x565194c04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c04b70 0x565194c04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 -- 172.21.15.3:0/1085455508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 msgr2=0x565194c0ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c0ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 -- 172.21.15.3:0/1085455508 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565194b0f340 con 0x565194c054b0 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.730+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b800aa4d0 tx=0x7f9b800ad750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b30c62f31996fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b78bfe640 1 -- 172.21.15.3:0/1085455508 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800deeb0 con 0x565194c054b0 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b78bfe640 1 -- 172.21.15.3:0/1085455508 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b800aa170 con 0x565194c054b0 2024-09-12T02:22:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b78bfe640 1 -- 172.21.15.3:0/1085455508 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800acc20 con 0x565194c054b0 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1085455508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 msgr2=0x565194c09f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b800aa4d0 tx=0x7f9b800ad750 comp rx=0 tx=0).stop 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1085455508 shutdown_connections 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c0ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c09f70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1085455508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c04b70 0x565194c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.731+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1085455508 >> 172.21.15.3:0/1085455508 conn(0x565194c001e0 msgr2=0x565194c005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1085455508 shutdown_connections 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1085455508 wait complete. 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- start start 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 Processor -- start 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c054b0 0x565194c0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565194c0d4b0 con 0x565194c04b70 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565194af1290 con 0x565194c0a540 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565194c0d630 con 0x565194c054b0 2024-09-12T02:22:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8f1c4740 10 client.0 _ll_register_callbacks cb 0x565194b22140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60428/0 (socket says 172.21.15.3:60428) 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/7108985 learned_addr learned my addr 172.21.15.3:0/7108985 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b898e6640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c054b0 0x565194c0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.732+0000 7f9b8a8e8640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c054b0 msgr2=0x565194c0f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c054b0 0x565194c0f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/7108985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 msgr2=0x565194c162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/7108985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519487b210 con 0x565194c04b70 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400f010 tx=0x7f9b74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b779ced25f8576 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74025070 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9b740138a0 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b800aa170 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9b74031050 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8f1c4740 10 client.16338 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565194947620 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.733+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b800aa170 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f9b740203e0 con 0x565194c04b70 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 10 client.16338 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 20 client.16338 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:37.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c1cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 4 client.16338 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565194b75a20 con 0x565194c1ab20 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b898e6640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c1cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b898e6640 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c1cf30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c792dee5d8a09f4 server_cookie=d6030b70ebbf8dad in_seq=0 out_seq=0 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.734+0000 7f9b6dffb640 10 client.16338 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b6dffb640 1 -- 172.21.15.3:0/7108985 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565194b75a20 con 0x565194c1ab20 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b6dffb640 10 client.16338 handle_command_reply: tid=0 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 client.16338 shutdown 2024-09-12T02:22:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 msgr2=0x565194c0ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400f010 tx=0x7f9b74004770 comp rx=0 tx=0).stop 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 shutdown_connections 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c1cf30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b6dffb640 0 client.16338 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c0a540 0x565194c162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c054b0 0x565194c0f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/7108985 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c0ed70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 >> 172.21.15.3:0/7108985 conn(0x565194c001e0 msgr2=0x565194c019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 shutdown_connections 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/7108985 wait complete. 2024-09-12T02:22:37.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.735+0000 7f9b8f1c4740 20 client.16338 trim_cache size 0 max 16384 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 Processor -- start 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 -- start start 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 0x565194c0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c16280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800e460 con 0x565194c054b0 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800ffa0 con 0x565194c04b70 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800eb30 con 0x565194c0a540 2024-09-12T02:22:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60434/0 (socket says 172.21.15.3:60434) 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 -- 172.21.15.3:0/3952472062 learned_addr learned my addr 172.21.15.3:0/3952472062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 0x565194c0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8a8e8640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c16280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 -- 172.21.15.3:0/3952472062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 msgr2=0x565194c16280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c16280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 -- 172.21.15.3:0/3952472062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 msgr2=0x565194c0f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 0x565194c0f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 -- 172.21.15.3:0/3952472062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565194c00000 con 0x565194c054b0 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 0x565194c0f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.750+0000 7f9b898e6640 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b800a9f60 tx=0x7f9b800b1ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b11923d10eb431d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b6ffff640 1 -- 172.21.15.3:0/3952472062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800acf50 con 0x565194c054b0 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b6ffff640 1 -- 172.21.15.3:0/3952472062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b800abe00 con 0x565194c054b0 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b6ffff640 1 -- 172.21.15.3:0/3952472062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800acc20 con 0x565194c054b0 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/3952472062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 msgr2=0x565194c04f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9b800a9f60 tx=0x7f9b800b1ff0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/3952472062 shutdown_connections 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c16280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c054b0 0x565194c04f50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/3952472062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c04b70 0x565194c0f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/3952472062 >> 172.21.15.3:0/3952472062 conn(0x565194c001e0 msgr2=0x565194bfea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/3952472062 shutdown_connections 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/3952472062 wait complete. 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.751+0000 7f9b8f1c4740 1 -- start start 2024-09-12T02:22:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 Processor -- start 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 0x565194c13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800f8d0 con 0x565194c04b70 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800f200 con 0x565194c054b0 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6800c440 con 0x565194c0a540 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 10 client.0 _ll_register_callbacks cb 0x7f9b7400ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60442/0 (socket says 172.21.15.3:60442) 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/1484231961 learned_addr learned my addr 172.21.15.3:0/1484231961 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a8e8640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 0x565194c13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 msgr2=0x565194c14b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c14b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/1484231961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 msgr2=0x565194c13b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 0x565194c13b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 -- 172.21.15.3:0/1484231961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565194c15190 con 0x565194c04b70 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 0x565194c13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8a0e7640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006b00 tx=0x7f9b74009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657e2219e1ea9838 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74025040 con 0x565194c04b70 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9b74020a20 con 0x565194c04b70 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.752+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b800abe00 con 0x565194c04b70 2024-09-12T02:22:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9b7401b040 con 0x565194c04b70 2024-09-12T02:22:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 10 client.16344 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565194c16510 con 0x565194c04b70 2024-09-12T02:22:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b800abe00 con 0x565194c04b70 2024-09-12T02:22:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f9b74013c40 con 0x565194c04b70 2024-09-12T02:22:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 10 client.16344 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 20 client.16344 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 4 client.16344 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565194b6fd70 con 0x565194c1ab20 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.753+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.754+0000 7f9b898e6640 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c87e50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9889e8ce2a015f34 server_cookie=8a4c8f082754c9b7 in_seq=0 out_seq=0 2024-09-12T02:22:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:37.754+0000 7f9b6dffb640 10 client.16344 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:38.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:37 smithi003 ceph-mon[31476]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 96 MiB/s wr, 13.60k op/s 2024-09-12T02:22:38.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:37 smithi026 ceph-mon[35496]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 96 MiB/s wr, 13.60k op/s 2024-09-12T02:22:38.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:37 smithi092 ceph-mon[36089]: pgmap v463: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 96 MiB/s wr, 13.60k op/s 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.301+0000 7f9b6dffb640 1 -- 172.21.15.3:0/1484231961 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565194b6fd70 con 0x565194c1ab20 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.301+0000 7f9b6dffb640 10 client.16344 handle_command_reply: tid=0 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.301+0000 7f9b8f1c4740 1 client.16344 shutdown 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 msgr2=0x565194c13630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9b74006b00 tx=0x7f9b74009270 comp rx=0 tx=0).stop 2024-09-12T02:22:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 shutdown_connections 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565194c1ab20 0x565194c87e50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565194c0a540 0x565194c14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565194c054b0 0x565194c13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 --2- 172.21.15.3:0/1484231961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565194c04b70 0x565194c13630 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 >> 172.21.15.3:0/1484231961 conn(0x565194c001e0 msgr2=0x565194bff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b6dffb640 0 client.16344 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 shutdown_connections 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 1 -- 172.21.15.3:0/1484231961 wait complete. 2024-09-12T02:22:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.302+0000 7f9b8f1c4740 20 client.16344 trim_cache size 0 max 16384 2024-09-12T02:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:22:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 msgr2=0x7f9b700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 0x7f9b700a5260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009a30 tx=0x7f9b74009190 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 msgr2=0x7f9b84196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 0x7f9b84196020 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9b8001f9b0 tx=0x7f9b8001fe70 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 shutdown_connections 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f9b700a2da0 0x7f9b700a5260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b84105510 0x7f9b8419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b84102b60 0x7f9b84196020 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 --2- 172.21.15.3:0/3108349426 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b84100210 0x7f9b84195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.303+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 >> 172.21.15.3:0/3108349426 conn(0x7f9b840fc000 msgr2=0x7f9b840fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.304+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 shutdown_connections 2024-09-12T02:22:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:38.304+0000 7f9b8c34b640 1 -- 172.21.15.3:0/3108349426 wait complete. 2024-09-12T02:22:40.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:39 smithi026 ceph-mon[35496]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 92 MiB/s wr, 12.93k op/s 2024-09-12T02:22:40.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:39 smithi003 ceph-mon[31476]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 92 MiB/s wr, 12.93k op/s 2024-09-12T02:22:40.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:39 smithi092 ceph-mon[36089]: pgmap v464: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 92 MiB/s wr, 12.93k op/s 2024-09-12T02:22:42.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:41 smithi026 ceph-mon[35496]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 84 MiB/s wr, 11.71k op/s 2024-09-12T02:22:42.277 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:41 smithi003 ceph-mon[31476]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 84 MiB/s wr, 11.71k op/s 2024-09-12T02:22:42.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:41 smithi092 ceph-mon[36089]: pgmap v465: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 84 MiB/s wr, 11.71k op/s 2024-09-12T02:22:44.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:43 smithi026 ceph-mon[35496]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 98 MiB/s wr, 13.88k op/s 2024-09-12T02:22:44.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:43 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:22:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:43 smithi003 ceph-mon[31476]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 98 MiB/s wr, 13.88k op/s 2024-09-12T02:22:44.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:43 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:22:44.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:43 smithi092 ceph-mon[36089]: pgmap v466: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 98 MiB/s wr, 13.88k op/s 2024-09-12T02:22:44.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:43 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:22:46.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:45 smithi026 ceph-mon[35496]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 89 MiB/s wr, 12.85k op/s 2024-09-12T02:22:46.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:45 smithi003 ceph-mon[31476]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 89 MiB/s wr, 12.85k op/s 2024-09-12T02:22:46.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:45 smithi092 ceph-mon[36089]: pgmap v467: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 89 MiB/s wr, 12.85k op/s 2024-09-12T02:22:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:47 smithi003 ceph-mon[31476]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 97 MiB/s wr, 14.39k op/s 2024-09-12T02:22:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:47 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:22:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:47 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:22:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:47 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:22:48.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:47 smithi026 ceph-mon[35496]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 97 MiB/s wr, 14.39k op/s 2024-09-12T02:22:48.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:47 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:22:48.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:47 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:22:48.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:47 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:22:48.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:47 smithi092 ceph-mon[36089]: pgmap v468: 97 pgs: 97 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 97 MiB/s wr, 14.39k op/s 2024-09-12T02:22:48.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:47 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:22:48.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:47 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:22:48.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:47 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:22:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:50 smithi026 ceph-mon[35496]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 91 MiB/s wr, 14.04k op/s 2024-09-12T02:22:50.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:49 smithi003 ceph-mon[31476]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 91 MiB/s wr, 14.04k op/s 2024-09-12T02:22:50.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:49 smithi092 ceph-mon[36089]: pgmap v469: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 91 MiB/s wr, 14.04k op/s 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 -- 172.21.15.3:0/2732098472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50102610 msgr2=0x7efe50108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 --2- 172.21.15.3:0/2732098472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50102610 0x7efe50108ea0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efe48009ca0 tx=0x7efe4802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 -- 172.21.15.3:0/2732098472 shutdown_connections 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 --2- 172.21.15.3:0/2732098472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50102610 0x7efe50108ea0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 --2- 172.21.15.3:0/2732098472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50101c50 0x7efe501020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 --2- 172.21.15.3:0/2732098472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50100a60 0x7efe50100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 -- 172.21.15.3:0/2732098472 >> 172.21.15.3:0/2732098472 conn(0x7efe500fc1c0 msgr2=0x7efe500fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:51.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 -- 172.21.15.3:0/2732098472 shutdown_connections 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.442+0000 7efe58586640 1 -- 172.21.15.3:0/2732098472 wait complete. 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.443+0000 7efe58586640 1 Processor -- start 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.443+0000 7efe58586640 1 -- start start 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.443+0000 7efe58586640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50100a60 0x7efe50197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.443+0000 7efe58586640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.443+0000 7efe58586640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50102610 0x7efe5019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe58586640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010d890 con 0x7efe50101c50 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe58586640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010d710 con 0x7efe50100a60 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe58586640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010da10 con 0x7efe50102610 2024-09-12T02:22:51.446 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33694/0 (socket says 172.21.15.3:33694) 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 -- 172.21.15.3:0/3491042747 learned_addr learned my addr 172.21.15.3:0/3491042747 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe56322640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50100a60 0x7efe50197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe56b23640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50102610 0x7efe5019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 -- 172.21.15.3:0/3491042747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50102610 msgr2=0x7efe5019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50102610 0x7efe5019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 -- 172.21.15.3:0/3491042747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50100a60 msgr2=0x7efe50197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50100a60 0x7efe50197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 -- 172.21.15.3:0/3491042747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5019fa40 con 0x7efe50101c50 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe55b21640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efe4000d950 tx=0x7efe4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.447 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe477fe640 1 -- 172.21.15.3:0/3491042747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40017070 con 0x7efe50101c50 2024-09-12T02:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.444+0000 7efe477fe640 1 -- 172.21.15.3:0/3491042747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe40009c30 con 0x7efe50101c50 2024-09-12T02:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.445+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5019fd30 con 0x7efe50101c50 2024-09-12T02:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.445+0000 7efe477fe640 1 -- 172.21.15.3:0/3491042747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4000f6b0 con 0x7efe50101c50 2024-09-12T02:22:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.445+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe501a0220 con 0x7efe50101c50 2024-09-12T02:22:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.447+0000 7efe5b3fd740 1 Processor -- start 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.447+0000 7efe5b3fd740 1 -- start start 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdafda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b58bd0e2f0 con 0x55b58bdaa9a0 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b58bcb6270 con 0x55b58bdb0370 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b58bd0a2b0 con 0x55b58bdab2e0 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe477fe640 1 -- 172.21.15.3:0/3491042747 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efe400109d0 con 0x7efe50101c50 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33710/0 (socket says 172.21.15.3:33710) 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 -- 172.21.15.3:0/1308367299 learned_addr learned my addr 172.21.15.3:0/1308367299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56322640 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdafda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 -- 172.21.15.3:0/1308367299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 msgr2=0x55b58bdafda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdafda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 -- 172.21.15.3:0/1308367299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 msgr2=0x55b58bdb2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdb2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 -- 172.21.15.3:0/1308367299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58bcb5340 con 0x55b58bdaa9a0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.448+0000 7efe56b23640 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe48031c10 tx=0x7efe48005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2159e450a4ad9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe477fe640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 0x7efe3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe44bfe640 1 -- 172.21.15.3:0/1308367299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48031650 con 0x55b58bdaa9a0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe44bfe640 1 -- 172.21.15.3:0/1308367299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48036040 con 0x55b58bdaa9a0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe44bfe640 1 -- 172.21.15.3:0/1308367299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48004710 con 0x55b58bdaa9a0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe477fe640 1 -- 172.21.15.3:0/3491042747 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe400ceca0 con 0x7efe50101c50 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe56322640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 0x7efe3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.456 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 -- 172.21.15.3:0/1308367299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 msgr2=0x55b58bdaada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe56322640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 0x7efe3c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55b58bdabd10 tx=0x7efe4c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efe48031c10 tx=0x7efe48005ef0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 -- 172.21.15.3:0/1308367299 shutdown_connections 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdb2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdafda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.449+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/1308367299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaada0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 -- 172.21.15.3:0/1308367299 >> 172.21.15.3:0/1308367299 conn(0x55b58bda6010 msgr2=0x55b58bda6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 -- 172.21.15.3:0/1308367299 shutdown_connections 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 -- 172.21.15.3:0/1308367299 wait complete. 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 -- start start 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 Processor -- start 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bdbc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.450+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b58bdb32e0 con 0x55b58bdab2e0 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b58bc97290 con 0x55b58bdaa9a0 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b58bdb3460 con 0x55b58bdb0370 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33722/0 (socket says 172.21.15.3:33722) 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 -- 172.21.15.3:0/4196799709 learned_addr learned my addr 172.21.15.3:0/4196799709 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 10 client.0 _ll_register_callbacks cb 0x55b58bcc8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe56b23640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bdbc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 -- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 msgr2=0x55b58bdbc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bdbc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 -- 172.21.15.3:0/4196799709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 msgr2=0x55b58bdb4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 -- 172.21.15.3:0/4196799709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58ba21210 con 0x55b58bdab2e0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe55b21640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe40097950 tx=0x7efe4009b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f17ddeb93b0719 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40020040 con 0x55b58bdab2e0 2024-09-12T02:22:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe4009a940 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe48036040 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe4009b4f0 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 10 client.16356 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.451+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b58baed620 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48036040 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7efe400972d0 con 0x55b58bdab2e0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 10 client.16356 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 20 client.16356 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58bdc2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 4 client.16356 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b58bd1ba20 con 0x55b58bdc08c0 2024-09-12T02:22:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.452+0000 7efe56322640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58bdc2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.453+0000 7efe39ffb640 10 client.16356 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.453+0000 7efe56322640 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58bdc2cd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa6bf5c28211a3c server_cookie=e4b95a3776c2374b in_seq=0 out_seq=0 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe39ffb640 1 -- 172.21.15.3:0/4196799709 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b58bd1ba20 con 0x55b58bdc08c0 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe39ffb640 10 client.16356 handle_command_reply: tid=0 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 client.16356 shutdown 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 msgr2=0x55b58bdb51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7efe40097950 tx=0x7efe4009b130 comp rx=0 tx=0).stop 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 shutdown_connections 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58bdc2cd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bdbc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe39ffb640 0 client.16356 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdb51a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/4196799709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 >> 172.21.15.3:0/4196799709 conn(0x55b58bda6010 msgr2=0x55b58bda7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 shutdown_connections 2024-09-12T02:22:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 1 -- 172.21.15.3:0/4196799709 wait complete. 2024-09-12T02:22:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.454+0000 7efe5b3fd740 20 client.16356 trim_cache size 0 max 16384 2024-09-12T02:22:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 Processor -- start 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 -- start start 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bda50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe501a9010 con 0x55b58bdab2e0 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe501aab50 con 0x55b58bdaa9a0 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe501a96e0 con 0x55b58bdb0370 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.469+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33738/0 (socket says 172.21.15.3:33738) 2024-09-12T02:22:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 -- 172.21.15.3:0/2185557331 learned_addr learned my addr 172.21.15.3:0/2185557331 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe56b23640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bda50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe56322640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 -- 172.21.15.3:0/2185557331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 msgr2=0x55b58bda50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bda50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 -- 172.21.15.3:0/2185557331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 msgr2=0x55b58bdaad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 -- 172.21.15.3:0/2185557331 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58bda6ca0 con 0x55b58bdab2e0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe56b23640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bda50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe55b21640 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efe4009b1a0 tx=0x7efe40099a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f83cca48dffdfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe3bfff640 1 -- 172.21.15.3:0/2185557331 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40020070 con 0x55b58bdab2e0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe3bfff640 1 -- 172.21.15.3:0/2185557331 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe40015020 con 0x55b58bdab2e0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.470+0000 7efe3bfff640 1 -- 172.21.15.3:0/2185557331 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4009a940 con 0x55b58bdab2e0 2024-09-12T02:22:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2185557331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 msgr2=0x55b58bdbc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7efe4009b1a0 tx=0x7efe40099a00 comp rx=0 tx=0).stop 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2185557331 shutdown_connections 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdb0370 0x55b58bda50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdab2e0 0x55b58bdbc1a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2185557331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdaa9a0 0x55b58bdaad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2185557331 >> 172.21.15.3:0/2185557331 conn(0x55b58bda6010 msgr2=0x55b58bda49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2185557331 shutdown_connections 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2185557331 wait complete. 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- start start 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 Processor -- start 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe501aa480 con 0x55b58bdaa9a0 2024-09-12T02:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe501a9db0 con 0x55b58bdb0370 2024-09-12T02:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe501a6ff0 con 0x55b58bdab2e0 2024-09-12T02:22:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe5b3fd740 10 client.0 _ll_register_callbacks cb 0x7efe4000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.471+0000 7efe56322640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe55b21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:33746/0 (socket says 172.21.15.3:33746) 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe55b21640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:55178/0 (socket says 172.21.15.3:55178) 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 -- 172.21.15.3:0/2001222945 learned_addr learned my addr 172.21.15.3:0/2001222945 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56b23640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdbb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 -- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 msgr2=0x55b58bdba050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdba050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 -- 172.21.15.3:0/2001222945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 msgr2=0x55b58bdbb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdbb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 -- 172.21.15.3:0/2001222945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58bdbb670 con 0x55b58bdaa9a0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe56322640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55b58bda6e90 tx=0x7efe4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acd2ac8fc34ff4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c005020 con 0x55b58bdaa9a0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe4c0051c0 con 0x55b58bdaa9a0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe40015020 con 0x55b58bdaa9a0 2024-09-12T02:22:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe4c0053e0 con 0x55b58bdaa9a0 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe5b3fd740 10 client.16362 fetch_fsmap learned FSMap version 37 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.472+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b58be2b8f0 con 0x55b58bdaa9a0 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe40015020 con 0x55b58bdaa9a0 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7efe4c003bc0 con 0x55b58bdaa9a0 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 10 client.16362 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 20 client.16362 populate_metadata read hostname 'smithi003' 2024-09-12T02:22:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58be2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:22:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 4 client.16362 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:22:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b58bd15d70 con 0x55b58bdc08c0 2024-09-12T02:22:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe55b21640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58be2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:22:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe55b21640 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58be2e310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19469d20f0d1aa1e server_cookie=f82cb3389ca58888 in_seq=0 out_seq=0 2024-09-12T02:22:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:51.473+0000 7efe39ffb640 10 client.16362 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:52.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:52 smithi026 ceph-mon[35496]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 77 MiB/s wr, 12.07k op/s 2024-09-12T02:22:52.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:52 smithi003 ceph-mon[31476]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 77 MiB/s wr, 12.07k op/s 2024-09-12T02:22:52.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:52 smithi092 ceph-mon[36089]: pgmap v470: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 77 MiB/s wr, 12.07k op/s 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe39ffb640 1 -- 172.21.15.3:0/2001222945 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b58bd15d70 con 0x55b58bdc08c0 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe39ffb640 10 client.16362 handle_command_reply: tid=0 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe5b3fd740 1 client.16362 shutdown 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 msgr2=0x55b58bdb9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55b58bda6e90 tx=0x7efe4c004730 comp rx=0 tx=0).stop 2024-09-12T02:22:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.711+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 shutdown_connections 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b58bdc08c0 0x55b58be2e310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b58bdb0370 0x55b58bdbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b58bdab2e0 0x55b58bdba050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 --2- 172.21.15.3:0/2001222945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b58bdaa9a0 0x55b58bdb9b10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 >> 172.21.15.3:0/2001222945 conn(0x55b58bda6010 msgr2=0x55b58bda51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe39ffb640 0 client.16362 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 shutdown_connections 2024-09-12T02:22:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 1 -- 172.21.15.3:0/2001222945 wait complete. 2024-09-12T02:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.712+0000 7efe5b3fd740 20 client.16362 trim_cache size 0 max 16384 2024-09-12T02:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:22:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 msgr2=0x7efe3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 0x7efe3c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55b58bdabd10 tx=0x7efe4c005f90 comp rx=0 tx=0).stop 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 msgr2=0x7efe50198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efe4000d950 tx=0x7efe4000de10 comp rx=0 tx=0).stop 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 shutdown_connections 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe3c0a2b80 0x7efe3c0a5040 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55b58bdabd10 tx=0x7efe4c005f90 comp rx=0 tx=0).stop 2024-09-12T02:22:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe50102610 0x7efe5019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe50101c50 0x7efe50198360 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.713+0000 7efe58586640 1 --2- 172.21.15.3:0/3491042747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe50100a60 0x7efe50197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.714+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 >> 172.21.15.3:0/3491042747 conn(0x7efe500fc1c0 msgr2=0x7efe50104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.714+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 shutdown_connections 2024-09-12T02:22:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:22:52.714+0000 7efe58586640 1 -- 172.21.15.3:0/3491042747 wait complete. 2024-09-12T02:22:54.279 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:54 smithi003 ceph-mon[31476]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 94 MiB/s wr, 15.25k op/s 2024-09-12T02:22:54.288 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:54 smithi092 ceph-mon[36089]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 94 MiB/s wr, 15.25k op/s 2024-09-12T02:22:54.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:54 smithi026 ceph-mon[35496]: pgmap v471: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 94 MiB/s wr, 15.25k op/s 2024-09-12T02:22:55.041 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 487917447, 'ino': 1099511628409, 'path': '/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b'}, {'damage_type': 'backtrace', 'id': 4093916290, 'ino': 1099511628453, 'path': '/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/logs/refs/remotes/origin'}] 2024-09-12T02:22:55.042 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-09-12T02:22:56.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:56 smithi003 ceph-mon[31476]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 MiB/s wr, 13.33k op/s 2024-09-12T02:22:56.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:56 smithi092 ceph-mon[36089]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 MiB/s wr, 13.33k op/s 2024-09-12T02:22:56.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:56 smithi026 ceph-mon[35496]: pgmap v472: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 78 MiB/s wr, 13.33k op/s 2024-09-12T02:22:56.533 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T02:22:56.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T02:22:57.302 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:22:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:22:58 smithi092 ceph-mon[36089]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 82 MiB/s wr, 14.28k op/s 2024-09-12T02:22:58.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:22:58 smithi026 ceph-mon[35496]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 82 MiB/s wr, 14.28k op/s 2024-09-12T02:22:58.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:22:58 smithi003 ceph-mon[31476]: pgmap v473: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 82 MiB/s wr, 14.28k op/s 2024-09-12T02:23:00.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:00 smithi092 ceph-mon[36089]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 83 MiB/s wr, 14.94k op/s 2024-09-12T02:23:00.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:00 smithi026 ceph-mon[35496]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 83 MiB/s wr, 14.94k op/s 2024-09-12T02:23:00.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:00 smithi003 ceph-mon[31476]: pgmap v474: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 83 MiB/s wr, 14.94k op/s 2024-09-12T02:23:02.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:02 smithi026 ceph-mon[35496]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 66 MiB/s wr, 12.20k op/s 2024-09-12T02:23:02.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:02 smithi003 ceph-mon[31476]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 66 MiB/s wr, 12.20k op/s 2024-09-12T02:23:02.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:02 smithi092 ceph-mon[36089]: pgmap v475: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 66 MiB/s wr, 12.20k op/s 2024-09-12T02:23:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1115065960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 msgr2=0x7fcac0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1115065960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0107bc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009930 tx=0x7fcaa802a420 comp rx=0 tx=0).stop 2024-09-12T02:23:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1115065960 shutdown_connections 2024-09-12T02:23:04.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1115065960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1115065960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0107bc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1115065960 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 0x7fcac01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1115065960 >> 172.21.15.3:0/1115065960 conn(0x7fcac00fc7d0 msgr2=0x7fcac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1115065960 shutdown_connections 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.110+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1115065960 wait complete. 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 Processor -- start 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 -- start start 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 0x7fcac01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0112050 con 0x7fcac0103780 2024-09-12T02:23:04.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111ed0 con 0x7fcac0108300 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcac5d4a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01121d0 con 0x7fcac0102dd0 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43866/0 (socket says 172.21.15.3:43866) 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcabeffd640 1 -- 172.21.15.3:0/1814505501 learned_addr learned my addr 172.21.15.3:0/1814505501 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.111+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 0x7fcac01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabffff640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 -- 172.21.15.3:0/1814505501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 msgr2=0x7fcac01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 0x7fcac01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 -- 172.21.15.3:0/1814505501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 msgr2=0x7fcac019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 -- 172.21.15.3:0/1814505501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019d580 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabffff640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005940 tx=0x7fcaa80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabcff9640 1 -- 172.21.15.3:0/1814505501 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa803f030 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabcff9640 1 -- 172.21.15.3:0/1814505501 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8003ce0 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019d810 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcabcff9640 1 -- 172.21.15.3:0/1814505501 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8035070 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.112+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a5150 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabcff9640 1 -- 172.21.15.3:0/1814505501 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcaa8042030 con 0x7fcac0103780 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 Processor -- start 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 -- start start 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b72810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b75140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565166ac92a0 con 0x565166b63660 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565166a70b10 con 0x565166b63f80 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcac8123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565166ac5260 con 0x565166b72d50 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43878/0 (socket says 172.21.15.3:43878) 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabffff640 1 -- 172.21.15.3:0/2086030149 learned_addr learned my addr 172.21.15.3:0/2086030149 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabcff9640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 0x7fcaac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b72810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.115+0000 7fcabeffd640 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b75140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 0x7fcaac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 -- 172.21.15.3:0/2086030149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 msgr2=0x565166b75140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b75140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 -- 172.21.15.3:0/2086030149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 msgr2=0x565166b72810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b72810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 -- 172.21.15.3:0/2086030149 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56516683ab90 con 0x565166b63660 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabcff9640 1 -- 172.21.15.3:0/1814505501 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fcaa80ef450 con 0x7fcac0103780 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcab80156e0 tx=0x7fcab8015ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9954f570cc8d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcab5ffb640 1 -- 172.21.15.3:0/2086030149 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8010750 con 0x565166b63660 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcabffff640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 0x7fcaac0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c000 tx=0x7fcab801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcab5ffb640 1 -- 172.21.15.3:0/2086030149 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8010dd0 con 0x565166b63660 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.116+0000 7fcab5ffb640 1 -- 172.21.15.3:0/2086030149 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab801de10 con 0x565166b63660 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- 172.21.15.3:0/2086030149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 msgr2=0x565166b63a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcab80156e0 tx=0x7fcab8015ba0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- 172.21.15.3:0/2086030149 shutdown_connections 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b75140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b72810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 --2- 172.21.15.3:0/2086030149 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b63a40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- 172.21.15.3:0/2086030149 >> 172.21.15.3:0/2086030149 conn(0x565166b5fcf0 msgr2=0x565166b600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- 172.21.15.3:0/2086030149 shutdown_connections 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- 172.21.15.3:0/2086030149 wait complete. 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.117+0000 7fcac8123740 1 -- start start 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 Processor -- start 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63660 0x565166b6fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b72d50 0x565166b70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x565166b75920 con 0x565166b63f80 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565166a516e0 con 0x565166b72d50 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565166b75aa0 con 0x565166b63660 2024-09-12T02:23:04.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcac8123740 10 client.0 _ll_register_callbacks cb 0x565166a75a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43880/0 (socket says 172.21.15.3:43880) 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 -- 172.21.15.3:0/1094397055 learned_addr learned my addr 172.21.15.3:0/1094397055 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63660 0x565166b6fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabffff640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b72d50 0x565166b70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 -- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63660 msgr2=0x565166b6fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63660 0x565166b6fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 -- 172.21.15.3:0/1094397055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b72d50 msgr2=0x565166b70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b72d50 0x565166b70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.118+0000 7fcabeffd640 1 -- 172.21.15.3:0/1094397055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565166b7ef30 con 0x565166b63f80 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80b80b0 tx=0x7fcaa8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484ad8b7b289dc4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:04.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8049070 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fcaa8003800 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab8010dd0 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcaa80bb600 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 10 client.16374 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565166b81880 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8010dd0 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa80ef0d0 con 0x565166b63f80 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 10 client.16374 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 10 client.16374 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 20 client.16374 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166b82c70 0x565166b85060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 4 client.16374 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:04.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.119+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565166ad69d0 con 0x565166b82c70 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.121+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166b82c70 0x565166b85060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.122+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166b82c70 0x565166b85060 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56842945a353ddee server_cookie=43293d0484263c4a in_seq=0 out_seq=0 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.122+0000 7fca9effd640 10 client.16374 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fca9effd640 1 -- 172.21.15.3:0/1094397055 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565166ad69d0 con 0x565166b82c70 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fca9effd640 10 client.16374 handle_command_reply: tid=0 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 client.16374 shutdown 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 msgr2=0x565166b70230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80b80b0 tx=0x7fcaa8009230 comp rx=0 tx=0).stop 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 shutdown_connections 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166b82c70 0x565166b85060 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b72d50 0x565166b70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63f80 0x565166b70230 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fca9effd640 0 client.16374 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 --2- 172.21.15.3:0/1094397055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63660 0x565166b6fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 >> 172.21.15.3:0/1094397055 conn(0x565166b5fcf0 msgr2=0x565166b65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 shutdown_connections 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 1 -- 172.21.15.3:0/1094397055 wait complete. 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.123+0000 7fcac8123740 20 client.16374 trim_cache size 0 max 16384 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 Processor -- start 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 -- start start 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b8a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b8add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ab330 con 0x565166b63660 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ace70 con 0x565166b63f80 2024-09-12T02:23:04.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcac8123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01aba00 con 0x565166b72d50 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.141+0000 7fcabf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43896/0 (socket says 172.21.15.3:43896) 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 -- 172.21.15.3:0/3857991490 learned_addr learned my addr 172.21.15.3:0/3857991490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabeffd640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b8a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabffff640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b8add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 -- 172.21.15.3:0/3857991490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 msgr2=0x565166b8add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b8add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 -- 172.21.15.3:0/3857991490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 msgr2=0x565166b8a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b8a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 -- 172.21.15.3:0/3857991490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565166b60e80 con 0x565166b63660 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabeffd640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000f80 tx=0x7fcab0012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6945935d0886e41c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcab57fa640 1 -- 172.21.15.3:0/3857991490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00118b0 con 0x565166b63660 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcab57fa640 1 -- 172.21.15.3:0/3857991490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab001e420 con 0x565166b63660 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.142+0000 7fcab57fa640 1 -- 172.21.15.3:0/3857991490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0025610 con 0x565166b63660 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- 172.21.15.3:0/3857991490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 msgr2=0x565166b658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000f80 tx=0x7fcab0012bb0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- 172.21.15.3:0/3857991490 shutdown_connections 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b72d50 0x565166b8add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63f80 0x565166b8a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 --2- 172.21.15.3:0/3857991490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b63660 0x565166b658f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- 172.21.15.3:0/3857991490 >> 172.21.15.3:0/3857991490 conn(0x565166b5fcf0 msgr2=0x565166b651e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- 172.21.15.3:0/3857991490 shutdown_connections 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- 172.21.15.3:0/3857991490 wait complete. 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.143+0000 7fcac8123740 1 -- start start 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 Processor -- start 2024-09-12T02:23:04.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63660 0x565166b772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63f80 0x565166b77830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac7a0 con 0x565166b72d50 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac0d0 con 0x565166b63660 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a9310 con 0x565166b63f80 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcac8123740 10 client.0 _ll_register_callbacks cb 0x7fcaa8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43908/0 (socket says 172.21.15.3:43908) 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 -- 172.21.15.3:0/1058852207 learned_addr learned my addr 172.21.15.3:0/1058852207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63660 0x565166b772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabeffd640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63f80 0x565166b77830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 -- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63f80 msgr2=0x565166b77830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63f80 0x565166b77830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 -- 172.21.15.3:0/1058852207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63660 msgr2=0x565166b772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.144+0000 7fcabffff640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63660 0x565166b772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcabffff640 1 -- 172.21.15.3:0/1058852207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565166be6e70 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcabffff640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcab8020000 tx=0x7fcab8015560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2e12151698825 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab801d9e0 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fcab801db80 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab001e420 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcab801dda0 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 10 client.16380 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565166bee1c0 con 0x565166b72d50 2024-09-12T02:23:04.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab001e420 con 0x565166b72d50 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fcab8025070 con 0x565166b72d50 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 10 client.16380 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 10 client.16380 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 20 client.16380 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166bee7a0 0x565166bf0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 4 client.16380 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.145+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x565166ad0d20 con 0x565166bee7a0 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.146+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166bee7a0 0x565166bf0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.146+0000 7fcabf7fe640 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166bee7a0 0x565166bf0bc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=561b31f44e6ded19 server_cookie=2559e5ed4ca5671e in_seq=0 out_seq=0 2024-09-12T02:23:04.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:03.146+0000 7fca9effd640 10 client.16380 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:04.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:04 smithi003 ceph-mon[31476]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 81 MiB/s wr, 14.87k op/s 2024-09-12T02:23:04.527 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:04 smithi026 ceph-mon[35496]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 81 MiB/s wr, 14.87k op/s 2024-09-12T02:23:04.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:04 smithi092 ceph-mon[36089]: pgmap v476: 97 pgs: 97 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 81 MiB/s wr, 14.87k op/s 2024-09-12T02:23:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.884+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1113070289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 msgr2=0x7fdf08119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.884+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1113070289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08119130 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009be0 tx=0x7fdf0002a6f0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1113070289 shutdown_connections 2024-09-12T02:23:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1113070289 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08119130 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1113070289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 0x7fdf08073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1113070289 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf08109700 0x7fdf08109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1113070289 >> 172.21.15.3:0/1113070289 conn(0x7fdf0806ecd0 msgr2=0x7fdf08071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.890+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1113070289 shutdown_connections 2024-09-12T02:23:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.891+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1113070289 wait complete. 2024-09-12T02:23:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.891+0000 7fdf0d3d8640 1 Processor -- start 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.893+0000 7fdf0d3d8640 1 -- start start 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 0x7fdf08115850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf08109700 0x7fdf081b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0811c680 con 0x7fdf080736b0 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0811c500 con 0x7fdf08072d10 2024-09-12T02:23:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.894+0000 7fdf0d3d8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0811c800 con 0x7fdf08109700 2024-09-12T02:23:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.895+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.895+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43936/0 (socket says 172.21.15.3:43936) 2024-09-12T02:23:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 -- 172.21.15.3:0/1170627376 learned_addr learned my addr 172.21.15.3:0/1170627376 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 0x7fdf08115850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 -- 172.21.15.3:0/1170627376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf08109700 msgr2=0x7fdf081b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf08109700 0x7fdf081b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 -- 172.21.15.3:0/1170627376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 msgr2=0x7fdf08115850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 0x7fdf08115850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.896+0000 7fdf067fc640 1 -- 172.21.15.3:0/1170627376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf08116430 con 0x7fdf080736b0 2024-09-12T02:23:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.899+0000 7fdf067fc640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdef000d950 tx=0x7fdef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:05.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.899+0000 7fdefffff640 1 -- 172.21.15.3:0/1170627376 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0017070 con 0x7fdf080736b0 2024-09-12T02:23:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.900+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf081b73f0 con 0x7fdf080736b0 2024-09-12T02:23:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.900+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081b7960 con 0x7fdf080736b0 2024-09-12T02:23:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.900+0000 7fdefffff640 1 -- 172.21.15.3:0/1170627376 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef0009c30 con 0x7fdf080736b0 2024-09-12T02:23:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.900+0000 7fdefffff640 1 -- 172.21.15.3:0/1170627376 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef000f620 con 0x7fdf080736b0 2024-09-12T02:23:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.904+0000 7fdefffff640 1 -- 172.21.15.3:0/1170627376 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdef000f7c0 con 0x7fdf080736b0 2024-09-12T02:23:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf10249740 1 Processor -- start 2024-09-12T02:23:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf10249740 1 -- start start 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf10249740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 0x5621eeb1bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf10249740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1c2e0 0x5621eeb20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf10249740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.908+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 0x5621eeb1bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 0x5621eeb1bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49094/0 (socket says 172.21.15.3:49094) 2024-09-12T02:23:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf077fe640 1 -- 172.21.15.3:0/2482092777 learned_addr learned my addr 172.21.15.3:0/2482092777 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1c2e0 0x5621eeb20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5621eea7f2f0 con 0x5621eeb21370 2024-09-12T02:23:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5621eea27270 con 0x5621eeb1c2e0 2024-09-12T02:23:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.909+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5621eea7b2b0 con 0x5621eeb1b9a0 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdefffff640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 0x7fdef40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 -- 172.21.15.3:0/2482092777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 msgr2=0x5621eeb1bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 0x5621eeb1bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 -- 172.21.15.3:0/2482092777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1c2e0 msgr2=0x5621eeb20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1c2e0 0x5621eeb20da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 -- 172.21.15.3:0/2482092777 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621eea26340 con 0x5621eeb21370 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb23830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdef00cc6b0 tx=0x7fdef0065fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7ce0ab2312198b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdf077fe640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 0x7fdef40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.910+0000 7fdefffff640 1 -- 172.21.15.3:0/1170627376 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdef00cec80 con 0x7fdf080736b0 2024-09-12T02:23:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.911+0000 7fdf077fe640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 0x7fdef40a5180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf00034040 tx=0x7fdf00009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.912+0000 7fdefd7fa640 1 -- 172.21.15.3:0/2482092777 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef00d3040 con 0x5621eeb21370 2024-09-12T02:23:05.913 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.912+0000 7fdefd7fa640 1 -- 172.21.15.3:0/2482092777 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef00664c0 con 0x5621eeb21370 2024-09-12T02:23:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 msgr2=0x5621eeb23830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb23830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdef00cc6b0 tx=0x7fdef0065fb0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 shutdown_connections 2024-09-12T02:23:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb23830 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1c2e0 0x5621eeb20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 --2- 172.21.15.3:0/2482092777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1b9a0 0x5621eeb1bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.913+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 >> 172.21.15.3:0/2482092777 conn(0x5621eeb17010 msgr2=0x5621eeb17410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.914+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 shutdown_connections 2024-09-12T02:23:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.915+0000 7fdf10249740 1 -- 172.21.15.3:0/2482092777 wait complete. 2024-09-12T02:23:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.915+0000 7fdf10249740 1 -- start start 2024-09-12T02:23:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.916+0000 7fdf10249740 1 Processor -- start 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb25b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5621eeb242e0 con 0x5621eeb1c2e0 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5621eea08290 con 0x5621eeb1b9a0 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5621eeb24460 con 0x5621eeb21370 2024-09-12T02:23:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.920+0000 7fdf10249740 10 client.0 _ll_register_callbacks cb 0x5621eea39140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.921+0000 7fdf06ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb25b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.921+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.921+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb2d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49096/0 (socket says 172.21.15.3:49096) 2024-09-12T02:23:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.921+0000 7fdf077fe640 1 -- 172.21.15.3:0/2038859256 learned_addr learned my addr 172.21.15.3:0/2038859256 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 -- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 msgr2=0x5621eeb2d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb2d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 -- 172.21.15.3:0/2038859256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 msgr2=0x5621eeb25b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb25b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 -- 172.21.15.3:0/2038859256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ee792210 con 0x5621eeb1c2e0 2024-09-12T02:23:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.923+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb26040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdef0005ce0 tx=0x7fdef0095de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b348ba060a42d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.924+0000 7fdeee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.924+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef006b070 con 0x5621eeb1c2e0 2024-09-12T02:23:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.925+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdef00cc790 con 0x5621eeb1c2e0 2024-09-12T02:23:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.925+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef00664c0 con 0x5621eeb1c2e0 2024-09-12T02:23:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.925+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdef0020050 con 0x5621eeb1c2e0 2024-09-12T02:23:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.926+0000 7fdf10249740 10 client.16392 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.926+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5621ee85e620 con 0x5621eeb1c2e0 2024-09-12T02:23:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.930+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef00664c0 con 0x5621eeb1c2e0 2024-09-12T02:23:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.930+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdef0097a50 con 0x5621eeb1c2e0 2024-09-12T02:23:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 10 client.16392 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 20 client.16392 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb313d0 0x5621eeb337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 4 client.16392 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.937+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621eea8ca20 con 0x5621eeb313d0 2024-09-12T02:23:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.938+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb313d0 0x5621eeb337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.943+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb313d0 0x5621eeb337f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79587e815c529aa6 server_cookie=97741daeb2c3887c in_seq=0 out_seq=0 2024-09-12T02:23:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.944+0000 7fdeee7fc640 10 client.16392 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.946+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2038859256 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621eea8ca20 con 0x5621eeb313d0 2024-09-12T02:23:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.946+0000 7fdeee7fc640 10 client.16392 handle_command_reply: tid=0 2024-09-12T02:23:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.947+0000 7fdf10249740 1 client.16392 shutdown 2024-09-12T02:23:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.950+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 msgr2=0x5621eeb26040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.950+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb26040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fdef0005ce0 tx=0x7fdef0095de0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 shutdown_connections 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb313d0 0x5621eeb337f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb2d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb26040 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 --2- 172.21.15.3:0/2038859256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb25b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.953+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 >> 172.21.15.3:0/2038859256 conn(0x5621eeb17010 msgr2=0x5621eeb18820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.954+0000 7fdeee7fc640 0 client.16392 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.963+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 shutdown_connections 2024-09-12T02:23:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.964+0000 7fdf10249740 1 -- 172.21.15.3:0/2038859256 wait complete. 2024-09-12T02:23:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:05.965+0000 7fdf10249740 20 client.16392 trim_cache size 0 max 16384 2024-09-12T02:23:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.004+0000 7fdf10249740 1 Processor -- start 2024-09-12T02:23:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.004+0000 7fdf10249740 1 -- start start 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.004+0000 7fdf10249740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb24890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.004+0000 7fdf10249740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf10249740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb24dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf10249740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800e4d0 con 0x5621eeb21370 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf10249740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdee8010010 con 0x5621eeb1b9a0 2024-09-12T02:23:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb24dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf10249740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800eba0 con 0x5621eeb1c2e0 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf06ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb24890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf067fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49110/0 (socket says 172.21.15.3:49110) 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.005+0000 7fdf067fc640 1 -- 172.21.15.3:0/1136008306 learned_addr learned my addr 172.21.15.3:0/1136008306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 -- 172.21.15.3:0/1136008306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 msgr2=0x5621eeb2d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 -- 172.21.15.3:0/1136008306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 msgr2=0x5621eeb24dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb24dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 -- 172.21.15.3:0/1136008306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621eeb17c90 con 0x5621eeb1b9a0 2024-09-12T02:23:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.006+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb24890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5621eeb34030 tx=0x7fdef8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea891a53f41e5afe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.008+0000 7fdefcff9640 1 -- 172.21.15.3:0/1136008306 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8016d10 con 0x5621eeb1b9a0 2024-09-12T02:23:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.008+0000 7fdefcff9640 1 -- 172.21.15.3:0/1136008306 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef800bc70 con 0x5621eeb1b9a0 2024-09-12T02:23:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.009+0000 7fdf10249740 1 -- 172.21.15.3:0/1136008306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 msgr2=0x5621eeb24890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.009+0000 7fdf10249740 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb24890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5621eeb34030 tx=0x7fdef8008040 comp rx=0 tx=0).stop 2024-09-12T02:23:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.010+0000 7fdf10249740 1 -- 172.21.15.3:0/1136008306 shutdown_connections 2024-09-12T02:23:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.010+0000 7fdf10249740 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb21370 0x5621eeb24dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.010+0000 7fdf10249740 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.010+0000 7fdf10249740 1 --2- 172.21.15.3:0/1136008306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb24890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.010+0000 7fdf10249740 1 -- 172.21.15.3:0/1136008306 >> 172.21.15.3:0/1136008306 conn(0x5621eeb17010 msgr2=0x5621eeb1d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.014+0000 7fdf10249740 1 -- 172.21.15.3:0/1136008306 shutdown_connections 2024-09-12T02:23:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.014+0000 7fdf10249740 1 -- 172.21.15.3:0/1136008306 wait complete. 2024-09-12T02:23:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.014+0000 7fdf10249740 1 -- start start 2024-09-12T02:23:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.014+0000 7fdf10249740 1 Processor -- start 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800f940 con 0x5621eeb1c2e0 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800f270 con 0x5621eeb1b9a0 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdee800c4b0 con 0x5621eeb21370 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf10249740 10 client.0 _ll_register_callbacks cb 0x7fdef0095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf077fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:49112/0 (socket says 172.21.15.3:49112) 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf077fe640 1 -- 172.21.15.3:0/2777420689 learned_addr learned my addr 172.21.15.3:0/2777420689 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.015+0000 7fdf067fc640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 -- 172.21.15.3:0/2777420689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 msgr2=0x5621eeb29f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb29f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 -- 172.21.15.3:0/2777420689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 msgr2=0x5621eeb2a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 -- 172.21.15.3:0/2777420689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621eeb9a5c0 con 0x5621eeb21370 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.016+0000 7fdf077fe640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdf00031000 tx=0x7fdf00030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=248e22724cb6f3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.017+0000 7fdeee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.017+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0004e070 con 0x5621eeb21370 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.017+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef800bc70 con 0x5621eeb21370 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.017+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdf00004040 con 0x5621eeb21370 2024-09-12T02:23:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.017+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdf00036040 con 0x5621eeb21370 2024-09-12T02:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.018+0000 7fdf10249740 10 client.25972 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.018+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5621eeb9b830 con 0x5621eeb21370 2024-09-12T02:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.019+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdef800bc70 con 0x5621eeb21370 2024-09-12T02:23:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.020+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fdf00031ac0 con 0x5621eeb21370 2024-09-12T02:23:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 10 client.25972 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 20 client.25972 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb9c1c0 0x5621eeb9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 4 client.25972 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:23:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.022+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621eea86d70 con 0x5621eeb9c1c0 2024-09-12T02:23:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.023+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb9c1c0 0x5621eeb9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.036+0000 7fdf06ffd640 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb9c1c0 0x5621eeb9e5e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70059f6542691c8 server_cookie=c26b0515f68d1246 in_seq=0 out_seq=0 2024-09-12T02:23:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:06.036+0000 7fdeee7fc640 10 client.25972 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:06.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:06 smithi026 ceph-mon[35496]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 MiB/s wr, 14.14k op/s 2024-09-12T02:23:06.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:06 smithi003 ceph-mon[31476]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 MiB/s wr, 14.14k op/s 2024-09-12T02:23:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:06 smithi092 ceph-mon[36089]: pgmap v477: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 76 MiB/s wr, 14.14k op/s 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdeee7fc640 1 -- 172.21.15.3:0/2777420689 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621eea86d70 con 0x5621eeb9c1c0 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdeee7fc640 10 client.25972 handle_command_reply: tid=0 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 client.25972 shutdown 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 msgr2=0x5621eeb9a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdf00031000 tx=0x7fdf00030a30 comp rx=0 tx=0).stop 2024-09-12T02:23:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 shutdown_connections 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5621eeb9c1c0 0x5621eeb9e5e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5621eeb21370 0x5621eeb9a080 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5621eeb1c2e0 0x5621eeb2a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 --2- 172.21.15.3:0/2777420689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5621eeb1b9a0 0x5621eeb29f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.369+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 >> 172.21.15.3:0/2777420689 conn(0x5621eeb17010 msgr2=0x5621eeb24890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.370+0000 7fdeee7fc640 0 client.25972 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.370+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 shutdown_connections 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.370+0000 7fdf10249740 1 -- 172.21.15.3:0/2777420689 wait complete. 2024-09-12T02:23:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.370+0000 7fdf10249740 20 client.25972 trim_cache size 0 max 16384 2024-09-12T02:23:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:23:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:23:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:23:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 msgr2=0x7fdef40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 0x7fdef40a5180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf00034040 tx=0x7fdf00009500 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 msgr2=0x7fdf08115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdef000d950 tx=0x7fdef000de10 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 shutdown_connections 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdef40a2cc0 0x7fdef40a5180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf08109700 0x7fdf081b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdf080736b0 0x7fdf08115d90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 --2- 172.21.15.3:0/1170627376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf08072d10 0x7fdf08115850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 >> 172.21.15.3:0/1170627376 conn(0x7fdf0806ecd0 msgr2=0x7fdf08071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 shutdown_connections 2024-09-12T02:23:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:07.371+0000 7fdf0d3d8640 1 -- 172.21.15.3:0/1170627376 wait complete. 2024-09-12T02:23:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:08 smithi026 ceph-mon[35496]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 72 MiB/s wr, 13.55k op/s 2024-09-12T02:23:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:08 smithi003 ceph-mon[31476]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 72 MiB/s wr, 13.55k op/s 2024-09-12T02:23:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:08 smithi092 ceph-mon[36089]: pgmap v478: 97 pgs: 97 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 72 MiB/s wr, 13.55k op/s 2024-09-12T02:23:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:10 smithi026 ceph-mon[35496]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 81 MiB/s wr, 15.49k op/s 2024-09-12T02:23:10.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:10 smithi003 ceph-mon[31476]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 81 MiB/s wr, 15.49k op/s 2024-09-12T02:23:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:10 smithi092 ceph-mon[36089]: pgmap v479: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 81 MiB/s wr, 15.49k op/s 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.465+0000 7fca9effd640 1 -- 172.21.15.3:0/1058852207 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565166ad0d20 con 0x565166bee7a0 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.465+0000 7fca9effd640 10 client.16380 handle_command_reply: tid=0 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.465+0000 7fcac8123740 1 client.16380 shutdown 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.465+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 msgr2=0x565166be6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.465+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcab8020000 tx=0x7fcab8015560 comp rx=0 tx=0).stop 2024-09-12T02:23:11.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 shutdown_connections 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565166bee7a0 0x565166bf0bc0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fca9effd640 0 client.16380 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565166b72d50 0x565166be6830 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565166b63f80 0x565166b77830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 --2- 172.21.15.3:0/1058852207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565166b63660 0x565166b772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 >> 172.21.15.3:0/1058852207 conn(0x565166b5fcf0 msgr2=0x565166b65a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 shutdown_connections 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 1 -- 172.21.15.3:0/1058852207 wait complete. 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.466+0000 7fcac8123740 20 client.16380 trim_cache size 0 max 16384 2024-09-12T02:23:11.468 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 37, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 10, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "704975bd": { 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 10, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:23:11.469 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.7, 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:11.470 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 msgr2=0x7fcaac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 0x7fcaac0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c000 tx=0x7fcab801b000 comp rx=0 tx=0).stop 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 msgr2=0x7fcac0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005940 tx=0x7fcaa80042f0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 shutdown_connections 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fcaac0a2c00 0x7fcaac0a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac0108300 0x7fcac019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcac0103780 0x7fcac0198b10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 --2- 172.21.15.3:0/1814505501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcac0102dd0 0x7fcac01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 >> 172.21.15.3:0/1814505501 conn(0x7fcac00fc7d0 msgr2=0x7fcac0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 shutdown_connections 2024-09-12T02:23:11.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:11.467+0000 7fcac5d4a640 1 -- 172.21.15.3:0/1814505501 wait complete. 2024-09-12T02:23:12.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:12 smithi026 ceph-mon[35496]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 69 MiB/s wr, 13.40k op/s 2024-09-12T02:23:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:12 smithi003 ceph-mon[31476]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 69 MiB/s wr, 13.40k op/s 2024-09-12T02:23:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:12 smithi092 ceph-mon[36089]: pgmap v480: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 69 MiB/s wr, 13.40k op/s 2024-09-12T02:23:14.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:14 smithi026 ceph-mon[35496]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 87 MiB/s wr, 16.92k op/s 2024-09-12T02:23:14.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:14 smithi003 ceph-mon[31476]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 87 MiB/s wr, 16.92k op/s 2024-09-12T02:23:14.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:14 smithi092 ceph-mon[36089]: pgmap v481: 97 pgs: 97 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 87 MiB/s wr, 16.92k op/s 2024-09-12T02:23:16.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:16 smithi026 ceph-mon[35496]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 78 MiB/s wr, 15.29k op/s 2024-09-12T02:23:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:16 smithi003 ceph-mon[31476]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 78 MiB/s wr, 15.29k op/s 2024-09-12T02:23:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:16 smithi092 ceph-mon[36089]: pgmap v482: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 78 MiB/s wr, 15.29k op/s 2024-09-12T02:23:18.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:18 smithi026 ceph-mon[35496]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 67 MiB/s wr, 13.19k op/s 2024-09-12T02:23:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:18 smithi003 ceph-mon[31476]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 67 MiB/s wr, 13.19k op/s 2024-09-12T02:23:18.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:18 smithi092 ceph-mon[36089]: pgmap v483: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 67 MiB/s wr, 13.19k op/s 2024-09-12T02:23:20.340 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:20 smithi092 ceph-mon[36089]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 64 MiB/s wr, 12.56k op/s 2024-09-12T02:23:20.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:20 smithi026 ceph-mon[35496]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 64 MiB/s wr, 12.56k op/s 2024-09-12T02:23:20.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:20 smithi003 ceph-mon[31476]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 64 MiB/s wr, 12.56k op/s 2024-09-12T02:23:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.534+0000 7fbada4f1640 1 -- 172.21.15.3:0/2254017635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 msgr2=0x7fbad41109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.534+0000 7fbada4f1640 1 --2- 172.21.15.3:0/2254017635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41109e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbac400baf0 tx=0x7fbac402acc0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.536+0000 7fbada4f1640 1 -- 172.21.15.3:0/2254017635 shutdown_connections 2024-09-12T02:23:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.536+0000 7fbada4f1640 1 --2- 172.21.15.3:0/2254017635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41109e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.536+0000 7fbada4f1640 1 --2- 172.21.15.3:0/2254017635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad4109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.536+0000 7fbada4f1640 1 --2- 172.21.15.3:0/2254017635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad4073b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.536+0000 7fbada4f1640 1 -- 172.21.15.3:0/2254017635 >> 172.21.15.3:0/2254017635 conn(0x7fbad406ef10 msgr2=0x7fbad4071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.539+0000 7fbada4f1640 1 -- 172.21.15.3:0/2254017635 shutdown_connections 2024-09-12T02:23:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.540+0000 7fbada4f1640 1 -- 172.21.15.3:0/2254017635 wait complete. 2024-09-12T02:23:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.540+0000 7fbada4f1640 1 Processor -- start 2024-09-12T02:23:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 -- start start 2024-09-12T02:23:20.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113e30 con 0x7fbad410a130 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113cb0 con 0x7fbad4109790 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.541+0000 7fbada4f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4113fb0 con 0x7fbad4073760 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.542+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.542+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad419e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36744/0 (socket says 172.21.15.3:36744) 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.542+0000 7fbad3fff640 1 -- 172.21.15.3:0/3804070963 learned_addr learned my addr 172.21.15.3:0/3804070963 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.542+0000 7fbad37fe640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.542+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.543+0000 7fbad37fe640 1 -- 172.21.15.3:0/3804070963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 msgr2=0x7fbad419e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.543+0000 7fbad37fe640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad419e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.543+0000 7fbad37fe640 1 -- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 msgr2=0x7fbad41b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.543+0000 7fbad37fe640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.543+0000 7fbad37fe640 1 -- 172.21.15.3:0/3804070963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41b7290 con 0x7fbad4109790 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.545+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.546+0000 7fbad37fe640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad419ea00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbac800af10 tx=0x7fbac800cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.547+0000 7fbad17fa640 1 -- 172.21.15.3:0/3804070963 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8007bb0 con 0x7fbad4109790 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.547+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41a1d00 con 0x7fbad4109790 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.548+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41a21f0 con 0x7fbad4109790 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.547+0000 7fbad17fa640 1 -- 172.21.15.3:0/3804070963 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac8007d50 con 0x7fbad4109790 2024-09-12T02:23:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.548+0000 7fbad17fa640 1 -- 172.21.15.3:0/3804070963 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8031460 con 0x7fbad4109790 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.555+0000 7fbadd28d740 1 Processor -- start 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.556+0000 7fbadd28d740 1 -- start start 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.556+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.556+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55da7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55da7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55da7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45634/0 (socket says 172.21.15.3:45634) 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:46334/0 (socket says 172.21.15.3:46334) 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55daa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbad3fff640 1 -- 172.21.15.3:0/1645291891 learned_addr learned my addr 172.21.15.3:0/1645291891 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd55d062f0 con 0x55bd55da32e0 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.557+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd55cae270 con 0x55bd55da29a0 2024-09-12T02:23:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.558+0000 7fbad17fa640 1 -- 172.21.15.3:0/3804070963 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbac8004f10 con 0x7fbad4109790 2024-09-12T02:23:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.560+0000 7fbad17fa640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 0x7fbabc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.560+0000 7fbad17fa640 1 -- 172.21.15.3:0/3804070963 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbac80f0b50 con 0x7fbad4109790 2024-09-12T02:23:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.560+0000 7fbad37fe640 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55daa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.560+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd55d022b0 con 0x55bd55da8370 2024-09-12T02:23:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.564+0000 7fbad37fe640 1 -- 172.21.15.3:0/1645291891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 msgr2=0x55bd55da2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.564+0000 7fbad37fe640 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.564+0000 7fbad37fe640 1 -- 172.21.15.3:0/1645291891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 msgr2=0x55bd55da7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.564+0000 7fbad37fe640 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55da7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.564+0000 7fbad37fe640 1 -- 172.21.15.3:0/1645291891 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd55cad340 con 0x55bd55da8370 2024-09-12T02:23:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.565+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.565+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 0x7fbabc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.566+0000 7fbad37fe640 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55daa830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbac80bd040 tx=0x7fbac80b9710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b15f572b127cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.568+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 0x7fbabc0a5250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbac4035040 tx=0x7fbac4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.568+0000 7fbaba7fc640 1 -- 172.21.15.3:0/1645291891 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac803f040 con 0x55bd55da8370 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.568+0000 7fbaba7fc640 1 -- 172.21.15.3:0/1645291891 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac80c5fb0 con 0x55bd55da8370 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.568+0000 7fbaba7fc640 1 -- 172.21.15.3:0/1645291891 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac80bc9f0 con 0x55bd55da8370 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.569+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 msgr2=0x55bd55daa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.569+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55daa830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbac80bd040 tx=0x7fbac80b9710 comp rx=0 tx=0).stop 2024-09-12T02:23:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.572+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 shutdown_connections 2024-09-12T02:23:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.572+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55daa830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.572+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55da7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.572+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1645291891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55da2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.572+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 >> 172.21.15.3:0/1645291891 conn(0x55bd55d9e010 msgr2=0x55bd55d9e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.574+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 shutdown_connections 2024-09-12T02:23:20.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.576+0000 7fbadd28d740 1 -- 172.21.15.3:0/1645291891 wait complete. 2024-09-12T02:23:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 -- start start 2024-09-12T02:23:20.578 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 Processor -- start 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55dacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55dad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55bd55dab2e0 con 0x55bd55da8370 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55bd55c8f290 con 0x55bd55da29a0 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.577+0000 7fbadd28d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bd55dab460 con 0x55bd55da32e0 2024-09-12T02:23:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.578+0000 7fbadd28d740 10 client.0 _ll_register_callbacks cb 0x55bd55cc0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.578+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.578+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45638/0 (socket says 172.21.15.3:45638) 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.578+0000 7fbad8a8e640 1 -- 172.21.15.3:0/3395303299 learned_addr learned my addr 172.21.15.3:0/3395303299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.578+0000 7fbad37fe640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55dad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 -- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 msgr2=0x55bd55dad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55dad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 -- 172.21.15.3:0/3395303299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 msgr2=0x55bd55dacbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55dacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 -- 172.21.15.3:0/3395303299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd55a19210 con 0x55bd55da8370 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbac4034000 tx=0x7fbac4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd80dd6258b9267 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.579+0000 7fbab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.581+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac400b810 con 0x55bd55da8370 2024-09-12T02:23:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.581+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbac4004050 con 0x55bd55da8370 2024-09-12T02:23:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.583+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac80c5fb0 con 0x55bd55da8370 2024-09-12T02:23:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.584+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbac4004250 con 0x55bd55da8370 2024-09-12T02:23:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.584+0000 7fbadd28d740 10 client.16410 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.585+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bd55ae5620 con 0x55bd55da8370 2024-09-12T02:23:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.586+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac80c5fb0 con 0x55bd55da8370 2024-09-12T02:23:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.586+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fbac4003790 con 0x55bd55da8370 2024-09-12T02:23:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.587+0000 7fbadd28d740 10 client.16410 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.587+0000 7fbadd28d740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.587+0000 7fbadd28d740 20 client.16410 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.587+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55db8880 0x55bd55dbad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.587+0000 7fbadd28d740 4 client.16410 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.588+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd55d13a20 con 0x55bd55db8880 2024-09-12T02:23:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.592+0000 7fbad3fff640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55db8880 0x55bd55dbad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.594+0000 7fbad3fff640 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55db8880 0x55bd55dbad20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deec35facfea80b5 server_cookie=890ad42327b1933b in_seq=0 out_seq=0 2024-09-12T02:23:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.594+0000 7fbab37fe640 10 client.16410 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.595+0000 7fbab37fe640 1 -- 172.21.15.3:0/3395303299 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd55d13a20 con 0x55bd55db8880 2024-09-12T02:23:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.595+0000 7fbab37fe640 10 client.16410 handle_command_reply: tid=0 2024-09-12T02:23:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.595+0000 7fbadd28d740 1 client.16410 shutdown 2024-09-12T02:23:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.597+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 msgr2=0x55bd55db4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.597+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbac4034000 tx=0x7fbac4005b50 comp rx=0 tx=0).stop 2024-09-12T02:23:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 shutdown_connections 2024-09-12T02:23:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55db8880 0x55bd55dbad20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db4170 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55dad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 --2- 172.21.15.3:0/3395303299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55dacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.601+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 >> 172.21.15.3:0/3395303299 conn(0x55bd55d9e010 msgr2=0x55bd55d9f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.602+0000 7fbab37fe640 0 client.16410 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.603+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 shutdown_connections 2024-09-12T02:23:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.603+0000 7fbadd28d740 1 -- 172.21.15.3:0/3395303299 wait complete. 2024-09-12T02:23:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.603+0000 7fbadd28d740 20 client.16410 trim_cache size 0 max 16384 2024-09-12T02:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.634+0000 7fbadd28d740 1 Processor -- start 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 -- start start 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55d9eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55db4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbac000e5d0 con 0x55bd55da32e0 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0010110 con 0x55bd55da29a0 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbadd28d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbac000eca0 con 0x55bd55da8370 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55db4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55d9eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55d9eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:46344/0 (socket says 172.21.15.3:46344) 2024-09-12T02:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad8a8e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36788/0 (socket says 172.21.15.3:36788) 2024-09-12T02:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.636+0000 7fbad3fff640 1 -- 172.21.15.3:0/1887843637 learned_addr learned my addr 172.21.15.3:0/1887843637 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.637+0000 7fbad8a8e640 1 -- 172.21.15.3:0/1887843637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 msgr2=0x55bd55d9eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.637+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55d9eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.637+0000 7fbad8a8e640 1 -- 172.21.15.3:0/1887843637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 msgr2=0x55bd55db4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.637+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55db4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.637+0000 7fbad8a8e640 1 -- 172.21.15.3:0/1887843637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd55da37c0 con 0x55bd55da8370 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.638+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbac4003bb0 tx=0x7fbac40058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70f439c9a065fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.638+0000 7fbab9ffb640 1 -- 172.21.15.3:0/1887843637 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40053a0 con 0x55bd55da8370 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.638+0000 7fbab9ffb640 1 -- 172.21.15.3:0/1887843637 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4050070 con 0x55bd55da8370 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.639+0000 7fbab9ffb640 1 -- 172.21.15.3:0/1887843637 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4004a40 con 0x55bd55da8370 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.639+0000 7fbadd28d740 1 -- 172.21.15.3:0/1887843637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 msgr2=0x55bd55d9f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.639+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbac4003bb0 tx=0x7fbac40058f0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 -- 172.21.15.3:0/1887843637 shutdown_connections 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da8370 0x55bd55d9f400 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da32e0 0x55bd55db4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 --2- 172.21.15.3:0/1887843637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55d9eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 -- 172.21.15.3:0/1887843637 >> 172.21.15.3:0/1887843637 conn(0x55bd55d9e010 msgr2=0x55bd55dba660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 -- 172.21.15.3:0/1887843637 shutdown_connections 2024-09-12T02:23:20.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 -- 172.21.15.3:0/1887843637 wait complete. 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.641+0000 7fbadd28d740 1 -- start start 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 Processor -- start 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55db1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55db1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbac000fa40 con 0x55bd55da8370 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fbac000f370 con 0x55bd55da29a0 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbac000c5b0 con 0x55bd55da32e0 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbadd28d740 10 client.0 _ll_register_callbacks cb 0x7fbac4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55db1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55db1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:36794/0 (socket says 172.21.15.3:36794) 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbad37fe640 1 -- 172.21.15.3:0/2770667078 learned_addr learned my addr 172.21.15.3:0/2770667078 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.642+0000 7fbad3fff640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55db1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.643+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 -- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 msgr2=0x55bd55db1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55db1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 -- 172.21.15.3:0/2770667078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 msgr2=0x55bd55db1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55db1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 -- 172.21.15.3:0/2770667078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd55db2fa0 con 0x55bd55da8370 2024-09-12T02:23:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbad8a8e640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db2990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbac402be60 tx=0x7fbac40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534b8183b71b5832 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.644+0000 7fbab37fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.645+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40037f0 con 0x55bd55da8370 2024-09-12T02:23:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.645+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac4050070 con 0x55bd55da8370 2024-09-12T02:23:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.645+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fbac40047c0 con 0x55bd55da8370 2024-09-12T02:23:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.646+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbac40049c0 con 0x55bd55da8370 2024-09-12T02:23:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.648+0000 7fbadd28d740 10 client.16416 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.648+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bd55db42b0 con 0x55bd55da8370 2024-09-12T02:23:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac4050070 con 0x55bd55da8370 2024-09-12T02:23:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fbac40694e0 con 0x55bd55da8370 2024-09-12T02:23:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 10 client.16416 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 20 client.16416 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55e23b10 0x55bd55e25f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 4 client.16416 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:23:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.649+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd55d0dd70 con 0x55bd55e23b10 2024-09-12T02:23:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.650+0000 7fbad3fff640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55e23b10 0x55bd55e25f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.654+0000 7fbad3fff640 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55e23b10 0x55bd55e25f30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8355cbe55e257174 server_cookie=2fc9f8492a6fd46b in_seq=0 out_seq=0 2024-09-12T02:23:20.656 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.655+0000 7fbab37fe640 10 client.16416 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.672+0000 7fbab37fe640 1 -- 172.21.15.3:0/2770667078 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd55d0dd70 con 0x55bd55e23b10 2024-09-12T02:23:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.672+0000 7fbab37fe640 10 client.16416 handle_command_reply: tid=0 2024-09-12T02:23:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.672+0000 7fbadd28d740 1 client.16416 shutdown 2024-09-12T02:23:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.674+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 msgr2=0x55bd55db2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.674+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db2990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbac402be60 tx=0x7fbac40054a0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 shutdown_connections 2024-09-12T02:23:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55bd55e23b10 0x55bd55e25f30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55bd55da8370 0x55bd55db2990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bd55da32e0 0x55bd55db1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 --2- 172.21.15.3:0/2770667078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd55da29a0 0x55bd55db1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.675+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 >> 172.21.15.3:0/2770667078 conn(0x55bd55d9e010 msgr2=0x55bd55d9c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.676+0000 7fbab37fe640 0 client.16416 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.677+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 shutdown_connections 2024-09-12T02:23:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.679+0000 7fbadd28d740 1 -- 172.21.15.3:0/2770667078 wait complete. 2024-09-12T02:23:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.679+0000 7fbadd28d740 20 client.16416 trim_cache size 0 max 16384 2024-09-12T02:23:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:23:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:23:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:23:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.681+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 msgr2=0x7fbabc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.681+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 0x7fbabc0a5250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbac4035040 tx=0x7fbac4009260 comp rx=0 tx=0).stop 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.681+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 msgr2=0x7fbad419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.681+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad419ea00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbac800af10 tx=0x7fbac800cc90 comp rx=0 tx=0).stop 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 shutdown_connections 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fbabc0a2d90 0x7fbabc0a5250 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbad410a130 0x7fbad41b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad4109790 0x7fbad419ea00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 --2- 172.21.15.3:0/3804070963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbad4073760 0x7fbad419e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.682+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 >> 172.21.15.3:0/3804070963 conn(0x7fbad406ef10 msgr2=0x7fbad410cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.685+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 shutdown_connections 2024-09-12T02:23:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:20.685+0000 7fbada4f1640 1 -- 172.21.15.3:0/3804070963 wait complete. 2024-09-12T02:23:22.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '704975bd', quiesce took 65.5 seconds. Will release after: -40.8 2024-09-12T02:23:22.051 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:23:22.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:22 smithi003 ceph-mon[31476]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 9.35k op/s 2024-09-12T02:23:22.357 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:23:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:22 smithi026 ceph-mon[35496]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 9.35k op/s 2024-09-12T02:23:22.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:22 smithi092 ceph-mon[36089]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 9.35k op/s 2024-09-12T02:23:24.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:24 smithi003 ceph-mon[31476]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 9.39k op/s 2024-09-12T02:23:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:24 smithi026 ceph-mon[35496]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 9.39k op/s 2024-09-12T02:23:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:24 smithi092 ceph-mon[36089]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 MiB/s wr, 9.39k op/s 2024-09-12T02:23:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:26 smithi026 ceph-mon[35496]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 6.40k op/s 2024-09-12T02:23:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:26 smithi003 ceph-mon[31476]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 6.40k op/s 2024-09-12T02:23:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:26 smithi092 ceph-mon[36089]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 34 MiB/s wr, 6.40k op/s 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 -- 172.21.15.3:0/4010468722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 msgr2=0x7f451c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 --2- 172.21.15.3:0/4010468722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c10d950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4514009cd0 tx=0x7f451402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 -- 172.21.15.3:0/4010468722 shutdown_connections 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 --2- 172.21.15.3:0/4010468722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c10d950 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 --2- 172.21.15.3:0/4010468722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c100a80 0x7f451c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 --2- 172.21.15.3:0/4010468722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c106a80 0x7f451c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 -- 172.21.15.3:0/4010468722 >> 172.21.15.3:0/4010468722 conn(0x7f451c0fc7a0 msgr2=0x7f451c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 -- 172.21.15.3:0/4010468722 shutdown_connections 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.944+0000 7f4523d15640 1 -- 172.21.15.3:0/4010468722 wait complete. 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 Processor -- start 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 -- start start 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c100a80 0x7f451c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c106a80 0x7f451c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f451c110e80 con 0x7f451c101420 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f451c110d00 con 0x7f451c106a80 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.945+0000 7f4523d15640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f451c111000 con 0x7f451c100a80 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45690/0 (socket says 172.21.15.3:45690) 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 -- 172.21.15.3:0/1707401049 learned_addr learned my addr 172.21.15.3:0/1707401049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:28.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45222b2640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c106a80 0x7f451c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4521ab1640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c100a80 0x7f451c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:28.202 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:23:28.202 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":37,"btime":"2024-09-12T02:22:15:807901+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:22:15.541633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:23:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 -- 172.21.15.3:0/1707401049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c100a80 msgr2=0x7f451c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:28.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c100a80 0x7f451c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 -- 172.21.15.3:0/1707401049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c106a80 msgr2=0x7f451c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c106a80 0x7f451c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 -- 172.21.15.3:0/1707401049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c19d380 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f45212b0640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d950 tx=0x7f450c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c017070 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f450c00de80 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c19d670 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f620 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.946+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1a4f50 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.948+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c100ee0 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.949+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f450c009c30 con 0x7f451c101420 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.949+0000 7f4512ffd640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 0x7f45080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:28.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.950+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f450c0cec50 con 0x7f451c101420 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.950+0000 7f4521ab1640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 0x7f45080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.950+0000 7f4521ab1640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 0x7f45080a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f451800ab60 tx=0x7f451800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:26.953+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f450c09bba0 con 0x7f451c101420 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.168+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f451c199730 con 0x7f451c101420 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.168+0000 7f4512ffd640 1 -- 172.21.15.3:0/1707401049 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11946 (secure 0 0 0) 0x7f451c199730 con 0x7f451c101420 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 37 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 msgr2=0x7f45080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 0x7f45080a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f451800ab60 tx=0x7f451800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 msgr2=0x7f451c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d950 tx=0x7f450c00dd50 comp rx=0 tx=0).stop 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 shutdown_connections 2024-09-12T02:23:28.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f45080a2b80 0x7f45080a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f451c106a80 0x7f451c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f451c101420 0x7f451c198910 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 --2- 172.21.15.3:0/1707401049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f451c100a80 0x7f451c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 >> 172.21.15.3:0/1707401049 conn(0x7f451c0fc7a0 msgr2=0x7f451c06a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 shutdown_connections 2024-09-12T02:23:28.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:27.171+0000 7f4523d15640 1 -- 172.21.15.3:0/1707401049 wait complete. 2024-09-12T02:23:28.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:28 smithi026 ceph-mon[35496]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.24k op/s 2024-09-12T02:23:28.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:28 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1707401049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:23:28.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:28 smithi003 ceph-mon[31476]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.24k op/s 2024-09-12T02:23:28.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:28 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1707401049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:23:28.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:28 smithi092 ceph-mon[36089]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.24k op/s 2024-09-12T02:23:28.541 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:28 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1707401049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:23:30.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:30 smithi026 ceph-mon[35496]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 MiB/s wr, 4.43k op/s 2024-09-12T02:23:30.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:30 smithi003 ceph-mon[31476]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 MiB/s wr, 4.43k op/s 2024-09-12T02:23:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:30 smithi092 ceph-mon[36089]: pgmap v489: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 MiB/s wr, 4.43k op/s 2024-09-12T02:23:32.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:32 smithi026 ceph-mon[35496]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 22 MiB/s wr, 4.04k op/s 2024-09-12T02:23:32.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:32 smithi003 ceph-mon[31476]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 22 MiB/s wr, 4.04k op/s 2024-09-12T02:23:32.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:32 smithi092 ceph-mon[36089]: pgmap v490: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 22 MiB/s wr, 4.04k op/s 2024-09-12T02:23:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 -- 172.21.15.3:0/2307586003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184102d60 msgr2=0x7ff1841031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2307586003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184102d60 0x7ff1841031e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff178009900 tx=0x7ff17802a410 comp rx=0 tx=0).stop 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 -- 172.21.15.3:0/2307586003 shutdown_connections 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2307586003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184103720 0x7ff184109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2307586003 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184102d60 0x7ff1841031e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2307586003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184101b70 0x7ff184101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 -- 172.21.15.3:0/2307586003 >> 172.21.15.3:0/2307586003 conn(0x7ff1840fd2f0 msgr2=0x7ff1840ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 -- 172.21.15.3:0/2307586003 shutdown_connections 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.812+0000 7ff1896a5640 1 -- 172.21.15.3:0/2307586003 wait complete. 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 Processor -- start 2024-09-12T02:23:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 -- start start 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 0x7ff184197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 0x7ff1841983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d7f0 con 0x7ff184103720 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d670 con 0x7ff184101b70 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1896a5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff18410d970 con 0x7ff184102d60 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52480/0 (socket says 172.21.15.3:52480) 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 -- 172.21.15.3:0/2533878178 learned_addr learned my addr 172.21.15.3:0/2533878178 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff182ffd640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 0x7ff184197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1827fc640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 0x7ff1841983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 -- 172.21.15.3:0/2533878178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 msgr2=0x7ff1841983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 0x7ff1841983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 -- 172.21.15.3:0/2533878178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 msgr2=0x7ff184197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 0x7ff184197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1837fe640 1 -- 172.21.15.3:0/2533878178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18419fad0 con 0x7ff184103720 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff1827fc640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 0x7ff1841983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.813+0000 7ff182ffd640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 0x7ff184197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff1837fe640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00cad0 tx=0x7ff17c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff16bfff640 1 -- 172.21.15.3:0/2533878178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00ae20 con 0x7ff184103720 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff16bfff640 1 -- 172.21.15.3:0/2533878178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c005020 con 0x7ff184103720 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff16bfff640 1 -- 172.21.15.3:0/2533878178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00f6f0 con 0x7ff184103720 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18419fdc0 con 0x7ff184103720 2024-09-12T02:23:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.814+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1841a0280 con 0x7ff184103720 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.816+0000 7ff16bfff640 1 -- 172.21.15.3:0/2533878178 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff17c0051d0 con 0x7ff184103720 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 Processor -- start 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 -- start start 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e047da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e04f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e95dfab2f0 con 0x55e95e0482e0 2024-09-12T02:23:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e95df53270 con 0x55e95e0479a0 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff18c446740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e95dfa72b0 con 0x55e95e04d370 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff182ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52486/0 (socket says 172.21.15.3:52486) 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff182ffd640 1 -- 172.21.15.3:0/3562201200 learned_addr learned my addr 172.21.15.3:0/3562201200 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff1827fc640 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e04f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.817+0000 7ff16bfff640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 -- 172.21.15.3:0/3562201200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 msgr2=0x55e95e04f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff1837fe640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e04f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 -- 172.21.15.3:0/3562201200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 msgr2=0x55e95e047da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e047da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 -- 172.21.15.3:0/3562201200 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95df52340 con 0x55e95e0482e0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff16bfff640 1 -- 172.21.15.3:0/2533878178 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff17c0cf4c0 con 0x7ff184103720 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff182ffd640 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff174009cd0 tx=0x7ff1740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b51f5003614fea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff1837fe640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 0x7ff16c0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff17c098680 tx=0x7ff17c091190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff1697fa640 1 -- 172.21.15.3:0/3562201200 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff174012450 con 0x55e95e0482e0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff1697fa640 1 -- 172.21.15.3:0/3562201200 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174012ad0 con 0x55e95e0482e0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.818+0000 7ff1697fa640 1 -- 172.21.15.3:0/3562201200 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17400fcc0 con 0x55e95e0482e0 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- 172.21.15.3:0/3562201200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 msgr2=0x55e95e04cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff174009cd0 tx=0x7ff1740117b0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- 172.21.15.3:0/3562201200 shutdown_connections 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e04f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0482e0 0x55e95e04cda0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- 172.21.15.3:0/3562201200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e047da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- 172.21.15.3:0/3562201200 >> 172.21.15.3:0/3562201200 conn(0x55e95e043010 msgr2=0x55e95e043410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- 172.21.15.3:0/3562201200 shutdown_connections 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- 172.21.15.3:0/3562201200 wait complete. 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 -- start start 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 Processor -- start 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e051c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.819+0000 7ff18c446740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e052140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e95e050340 con 0x55e95e04d370 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e95df34290 con 0x55e95e0479a0 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e95e0504c0 con 0x55e95e0482e0 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 10 client.0 _ll_register_callbacks cb 0x55e95df65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52502/0 (socket says 172.21.15.3:52502) 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 -- 172.21.15.3:0/2920253155 learned_addr learned my addr 172.21.15.3:0/2920253155 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 -- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 msgr2=0x55e95e052140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e052140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 -- 172.21.15.3:0/2920253155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 msgr2=0x55e95e051c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e051c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 -- 172.21.15.3:0/2920253155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95dcbe210 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff1827fc640 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff178031610 tx=0x7ff178005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e625570fa25d612 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178035040 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff17803caa0 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.820+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff174012ad0 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff17803cca0 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 10 client.16434 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e95dd8a620 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff174012ad0 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff17803a040 con 0x55e95e04d370 2024-09-12T02:23:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 10 client.16434 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 20 client.16434 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e05d820 0x55e95e05fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 4 client.16434 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95dfb8a20 con 0x55e95e05d820 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.821+0000 7ff182ffd640 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e05d820 0x55e95e05fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.822+0000 7ff166ff5640 10 client.16434 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.822+0000 7ff182ffd640 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e05d820 0x55e95e05fc90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eaba848b4be6e67 server_cookie=31d4154a6e33fd72 in_seq=0 out_seq=0 2024-09-12T02:23:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.823+0000 7ff166ff5640 1 -- 172.21.15.3:0/2920253155 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e95dfb8a20 con 0x55e95e05d820 2024-09-12T02:23:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.823+0000 7ff166ff5640 10 client.16434 handle_command_reply: tid=0 2024-09-12T02:23:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.823+0000 7ff18c446740 1 client.16434 shutdown 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.823+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 msgr2=0x55e95e059180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.823+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff178031610 tx=0x7ff178005d50 comp rx=0 tx=0).stop 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 shutdown_connections 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e05d820 0x55e95e05fc90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e059180 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff166ff5640 0 client.16434 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e052140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 --2- 172.21.15.3:0/2920253155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e051c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 >> 172.21.15.3:0/2920253155 conn(0x55e95e043010 msgr2=0x55e95e044820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 shutdown_connections 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 1 -- 172.21.15.3:0/2920253155 wait complete. 2024-09-12T02:23:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.824+0000 7ff18c446740 20 client.16434 trim_cache size 0 max 16384 2024-09-12T02:23:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.861+0000 7ff18c446740 1 Processor -- start 2024-09-12T02:23:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 -- start start 2024-09-12T02:23:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e059140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e064590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841a9040 con 0x55e95e04d370 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841aab80 con 0x55e95e0479a0 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff18c446740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841a9710 con 0x55e95e0482e0 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.862+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40902/0 (socket says 172.21.15.3:40902) 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 -- 172.21.15.3:0/3834546923 learned_addr learned my addr 172.21.15.3:0/3834546923 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e064590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 -- 172.21.15.3:0/3834546923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 msgr2=0x55e95e059140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:23:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e059140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 -- 172.21.15.3:0/3834546923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 msgr2=0x55e95e064590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e064590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.863+0000 7ff1827fc640 1 -- 172.21.15.3:0/3834546923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95e0487c0 con 0x55e95e0479a0 2024-09-12T02:23:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.866+0000 7ff1827fc640 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff178001eb0 tx=0x7ff178031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ee1897f76a43b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.868+0000 7ff168ff9640 1 -- 172.21.15.3:0/3834546923 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178035040 con 0x55e95e0479a0 2024-09-12T02:23:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.868+0000 7ff168ff9640 1 -- 172.21.15.3:0/3834546923 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17803c8a0 con 0x55e95e0479a0 2024-09-12T02:23:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.868+0000 7ff168ff9640 1 -- 172.21.15.3:0/3834546923 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178004920 con 0x55e95e0479a0 2024-09-12T02:23:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.870+0000 7ff18c446740 1 -- 172.21.15.3:0/3834546923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 msgr2=0x55e95e0420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.870+0000 7ff18c446740 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff178001eb0 tx=0x7ff178031400 comp rx=0 tx=0).stop 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.876+0000 7ff18c446740 1 -- 172.21.15.3:0/3834546923 shutdown_connections 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.876+0000 7ff18c446740 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e04d370 0x55e95e064590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.876+0000 7ff18c446740 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e0482e0 0x55e95e059140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.876+0000 7ff18c446740 1 --2- 172.21.15.3:0/3834546923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0479a0 0x55e95e0420b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.876+0000 7ff18c446740 1 -- 172.21.15.3:0/3834546923 >> 172.21.15.3:0/3834546923 conn(0x55e95e043010 msgr2=0x55e95e0419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.877+0000 7ff18c446740 1 -- 172.21.15.3:0/3834546923 shutdown_connections 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.877+0000 7ff18c446740 1 -- 172.21.15.3:0/3834546923 wait complete. 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.877+0000 7ff18c446740 1 -- start start 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.877+0000 7ff18c446740 1 Processor -- start 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0482e0 0x55e95e057200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e058210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841aa4b0 con 0x55e95e0479a0 2024-09-12T02:23:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841a9de0 con 0x55e95e0482e0 2024-09-12T02:23:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff1841a7040 con 0x55e95e04d370 2024-09-12T02:23:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff18c446740 10 client.0 _ll_register_callbacks cb 0x7ff174005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff1827fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52524/0 (socket says 172.21.15.3:52524) 2024-09-12T02:23:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.878+0000 7ff1827fc640 1 -- 172.21.15.3:0/339569574 learned_addr learned my addr 172.21.15.3:0/339569574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.882+0000 7ff1827fc640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e058210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.882+0000 7ff182ffd640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0482e0 0x55e95e057200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.887+0000 7ff1827fc640 1 -- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 msgr2=0x55e95e058210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.887+0000 7ff1827fc640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e058210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.887+0000 7ff1827fc640 1 -- 172.21.15.3:0/339569574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0482e0 msgr2=0x55e95e057200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.887+0000 7ff1827fc640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0482e0 0x55e95e057200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.887+0000 7ff1827fc640 1 -- 172.21.15.3:0/339569574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95e058820 con 0x55e95e0479a0 2024-09-12T02:23:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.888+0000 7ff1827fc640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff178005e50 tx=0x7ff178005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99b196ae4d9d518 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.888+0000 7ff166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.894+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178035070 con 0x55e95e0479a0 2024-09-12T02:23:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.894+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff17803d650 con 0x55e95e0479a0 2024-09-12T02:23:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.894+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff17803c8a0 con 0x55e95e0479a0 2024-09-12T02:23:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.895+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff17803d850 con 0x55e95e0479a0 2024-09-12T02:23:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.895+0000 7ff18c446740 10 client.16440 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.895+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e95e0c8950 con 0x55e95e0479a0 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17803c8a0 con 0x55e95e0479a0 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff178061a50 con 0x55e95e0479a0 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff18c446740 10 client.16440 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff18c446740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff18c446740 20 client.16440 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.896+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e0c92b0 0x55e95e0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.897+0000 7ff18c446740 4 client.16440 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:23:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.897+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e95dfb2d70 con 0x55e95e0c92b0 2024-09-12T02:23:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.906+0000 7ff182ffd640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e0c92b0 0x55e95e0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.923+0000 7ff182ffd640 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e0c92b0 0x55e95e0c96c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b28fcbeb32fde42 server_cookie=29b3d777153edb79 in_seq=0 out_seq=0 2024-09-12T02:23:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:33.924+0000 7ff166ff5640 10 client.16440 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:34.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:34 smithi026 ceph-mon[35496]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 MiB/s wr, 4.24k op/s 2024-09-12T02:23:34.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:34 smithi003 ceph-mon[31476]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 MiB/s wr, 4.24k op/s 2024-09-12T02:23:34.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.532+0000 7ff166ff5640 1 -- 172.21.15.3:0/339569574 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e95dfb2d70 con 0x55e95e0c92b0 2024-09-12T02:23:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.532+0000 7ff166ff5640 10 client.16440 handle_command_reply: tid=0 2024-09-12T02:23:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.532+0000 7ff18c446740 1 client.16440 shutdown 2024-09-12T02:23:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 msgr2=0x55e95e056cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff178005e50 tx=0x7ff178005c90 comp rx=0 tx=0).stop 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 shutdown_connections 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e95e0c92b0 0x55e95e0c96c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e95e04d370 0x55e95e058210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e95e0482e0 0x55e95e057200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 --2- 172.21.15.3:0/339569574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e95e0479a0 0x55e95e056cc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 >> 172.21.15.3:0/339569574 conn(0x55e95e043010 msgr2=0x55e95e0421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff166ff5640 0 client.16440 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 shutdown_connections 2024-09-12T02:23:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 1 -- 172.21.15.3:0/339569574 wait complete. 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.533+0000 7ff18c446740 20 client.16440 trim_cache size 0 max 16384 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.534+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 msgr2=0x7ff16c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.534+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 0x7ff16c0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff17c098680 tx=0x7ff17c091190 comp rx=0 tx=0).stop 2024-09-12T02:23:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.534+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 msgr2=0x7ff18419f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.534+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00cad0 tx=0x7ff17c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 shutdown_connections 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff184103720 0x7ff18419f460 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff184102d60 0x7ff1841983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 --2- 172.21.15.3:0/2533878178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff184101b70 0x7ff184197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 >> 172.21.15.3:0/2533878178 conn(0x7ff1840fd2f0 msgr2=0x7ff184106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.537+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 shutdown_connections 2024-09-12T02:23:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:34.538+0000 7ff1896a5640 1 -- 172.21.15.3:0/2533878178 wait complete. 2024-09-12T02:23:34.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:34 smithi092 ceph-mon[36089]: pgmap v491: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 22 MiB/s wr, 4.24k op/s 2024-09-12T02:23:36.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:36 smithi026 ceph-mon[35496]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 3.59k op/s 2024-09-12T02:23:36.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:36 smithi003 ceph-mon[31476]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 3.59k op/s 2024-09-12T02:23:36.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:36 smithi092 ceph-mon[36089]: pgmap v492: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 19 MiB/s wr, 3.59k op/s 2024-09-12T02:23:37.200 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db --set-id 704975bd --release --await' 2024-09-12T02:23:37.200 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db --set-id 704975bd --release --await 2024-09-12T02:23:37.465 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:23:38.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:38 smithi026 ceph-mon[35496]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 18 MiB/s wr, 3.54k op/s 2024-09-12T02:23:38.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:38 smithi003 ceph-mon[31476]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 18 MiB/s wr, 3.54k op/s 2024-09-12T02:23:38.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:38 smithi092 ceph-mon[36089]: pgmap v493: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 18 MiB/s wr, 3.54k op/s 2024-09-12T02:23:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:40 smithi026 ceph-mon[35496]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 19 MiB/s wr, 3.87k op/s 2024-09-12T02:23:40.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:40 smithi003 ceph-mon[31476]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 19 MiB/s wr, 3.87k op/s 2024-09-12T02:23:40.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:40 smithi092 ceph-mon[36089]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 19 MiB/s wr, 3.87k op/s 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 -- 172.21.15.3:0/541848466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 msgr2=0x7f0e88106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/541848466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e88106e60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009930 tx=0x7f0e7802a420 comp rx=0 tx=0).stop 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 -- 172.21.15.3:0/541848466 shutdown_connections 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/541848466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88101420 0x7f0e8810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/541848466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88100a80 0x7f0e88100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/541848466 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e88106e60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 -- 172.21.15.3:0/541848466 >> 172.21.15.3:0/541848466 conn(0x7f0e880fc7a0 msgr2=0x7f0e880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 -- 172.21.15.3:0/541848466 shutdown_connections 2024-09-12T02:23:42.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 -- 172.21.15.3:0/541848466 wait complete. 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.512+0000 7f0e8e14b640 1 Processor -- start 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 -- start start 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88101420 0x7f0e881989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e8819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e88111f60 con 0x7f0e88106a80 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e88111de0 con 0x7f0e88101420 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e8e14b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881120e0 con 0x7f0e88100a80 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:35620/0 (socket says 172.21.15.3:35620) 2024-09-12T02:23:42.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e87fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e8819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 -- 172.21.15.3:0/1516072974 learned_addr learned my addr 172.21.15.3:0/1516072974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88101420 0x7f0e881989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 -- 172.21.15.3:0/1516072974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88101420 msgr2=0x7f0e881989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88101420 0x7f0e881989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 -- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 msgr2=0x7f0e8819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e8819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e877fe640 1 -- 172.21.15.3:0/1516072974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8819d430 con 0x7f0e88100a80 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.513+0000 7f0e87fff640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e8819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009230 tx=0x7f0e78004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e857fa640 1 -- 172.21.15.3:0/1516072974 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7803f030 con 0x7f0e88100a80 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e857fa640 1 -- 172.21.15.3:0/1516072974 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e78003ad0 con 0x7f0e88100a80 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8819d6c0 con 0x7f0e88100a80 2024-09-12T02:23:42.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e857fa640 1 -- 172.21.15.3:0/1516072974 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78035070 con 0x7f0e88100a80 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.514+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e881a5000 con 0x7f0e88100a80 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e857fa640 1 -- 172.21.15.3:0/1516072974 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0e78042030 con 0x7f0e88100a80 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 Processor -- start 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 -- start start 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e71440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e7e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d88dd6620 con 0x560d88e71980 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d88d7de50 con 0x560d88e71060 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e90524740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d88dd25e0 con 0x560d88e7c240 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60276/0 (socket says 172.21.15.3:60276) 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e877fe640 1 -- 172.21.15.3:0/2998421815 learned_addr learned my addr 172.21.15.3:0/2998421815 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e7e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e87fff640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e71440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.517+0000 7f0e857fa640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 0x7f0e740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 -- 172.21.15.3:0/2998421815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 msgr2=0x560d88e7e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e7e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 -- 172.21.15.3:0/2998421815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 msgr2=0x560d88e71440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e71440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 -- 172.21.15.3:0/2998421815 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d88d6ed40 con 0x560d88e71980 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e87fff640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e71440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e87fff640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 0x7f0e740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e857fa640 1 -- 172.21.15.3:0/1516072974 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0e780ef140 con 0x7f0e88100a80 2024-09-12T02:23:42.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e877fe640 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0e7807d450 tx=0x7f0e7804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebe0ea544706488 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e7dffb640 1 -- 172.21.15.3:0/2998421815 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78049070 con 0x560d88e71980 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e7dffb640 1 -- 172.21.15.3:0/2998421815 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e78036260 con 0x560d88e71980 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.518+0000 7f0e7dffb640 1 -- 172.21.15.3:0/2998421815 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780bb3a0 con 0x560d88e71980 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- 172.21.15.3:0/2998421815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 msgr2=0x560d88e7bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e87fff640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 0x7f0e740a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009fd0 tx=0x7f0e80004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0e7807d450 tx=0x7f0e7804b000 comp rx=0 tx=0).stop 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- 172.21.15.3:0/2998421815 shutdown_connections 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e7e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e7bd00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 --2- 172.21.15.3:0/2998421815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e71440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- 172.21.15.3:0/2998421815 >> 172.21.15.3:0/2998421815 conn(0x560d88e6d5f0 msgr2=0x560d88e6d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- 172.21.15.3:0/2998421815 shutdown_connections 2024-09-12T02:23:42.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- 172.21.15.3:0/2998421815 wait complete. 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.519+0000 7f0e90524740 1 -- start start 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 Processor -- start 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71060 0x560d88e79160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e79be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560d88e7ebd0 con 0x560d88e71980 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x560d88d5ef10 con 0x560d88e7c240 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x560d88e7ed50 con 0x560d88e71060 2024-09-12T02:23:42.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e90524740 10 client.0 _ll_register_callbacks cb 0x560d88d82db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 37, 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 12, 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "704975bd": { 2024-09-12T02:23:42.482 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 12, 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 144.7, 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:23:42.483 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:42.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:42.485 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:23:42.485 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:23:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60292/0 (socket says 172.21.15.3:60292) 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 -- 172.21.15.3:0/1669301131 learned_addr learned my addr 172.21.15.3:0/1669301131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71060 0x560d88e79160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e87fff640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e79be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 -- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71060 msgr2=0x560d88e79160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71060 0x560d88e79160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 -- 172.21.15.3:0/1669301131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 msgr2=0x560d88e79be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e79be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e7ffff640 1 -- 172.21.15.3:0/1669301131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d88ae5580 con 0x560d88e71980 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.520+0000 7f0e87fff640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:23:42.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000bad0 tx=0x7f0e7000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9b1432be2dc811 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70011440 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0e700115e0 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e78036260 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0e700117e0 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 10 client.16452 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560d88baf960 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e78036260 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0e70024050 con 0x560d88e71980 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 10 client.16452 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 10 client.16452 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 20 client.16452 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:42.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88e8bb80 0x560d88e8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 4 client.16452 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.521+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d88de3b90 con 0x560d88e8bb80 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.522+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88e8bb80 0x560d88e8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.522+0000 7f0e877fe640 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88e8bb80 0x560d88e8dfe0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5cea3a72b95ddc server_cookie=9480fdbbef22f037 in_seq=0 out_seq=0 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.522+0000 7f0e66ffd640 10 client.16452 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e66ffd640 1 -- 172.21.15.3:0/1669301131 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d88de3b90 con 0x560d88e8bb80 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e66ffd640 10 client.16452 handle_command_reply: tid=0 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 client.16452 shutdown 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 msgr2=0x560d88e796a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000bad0 tx=0x7f0e7000bf90 comp rx=0 tx=0).stop 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 shutdown_connections 2024-09-12T02:23:42.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88e8bb80 0x560d88e8dfe0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e796a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 --2- 172.21.15.3:0/1669301131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71060 0x560d88e79160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e66ffd640 0 client.16452 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 >> 172.21.15.3:0/1669301131 conn(0x560d88e6d5f0 msgr2=0x560d88e73320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 shutdown_connections 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 1 -- 172.21.15.3:0/1669301131 wait complete. 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.523+0000 7f0e90524740 20 client.16452 trim_cache size 0 max 16384 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.540+0000 7f0e90524740 1 Processor -- start 2024-09-12T02:23:42.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.540+0000 7f0e90524740 1 -- start start 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e73ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881ad690 con 0x560d88e71980 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881af1d0 con 0x560d88e71060 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e90524740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881add60 con 0x560d88e7c240 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60302/0 (socket says 172.21.15.3:60302) 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 -- 172.21.15.3:0/898888825 learned_addr learned my addr 172.21.15.3:0/898888825 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e877fe640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e73ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e87fff640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 -- 172.21.15.3:0/898888825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 msgr2=0x560d88e842b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e842b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 -- 172.21.15.3:0/898888825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 msgr2=0x560d88e73ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e73ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 -- 172.21.15.3:0/898888825 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d88e8e660 con 0x560d88e71980 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e877fe640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e87fff640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0e7001bc90 tx=0x7f0e7000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faa85b669b872fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.541+0000 7f0e7d7fa640 1 -- 172.21.15.3:0/898888825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7001bd10 con 0x560d88e71980 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e7d7fa640 1 -- 172.21.15.3:0/898888825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70011440 con 0x560d88e71980 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e7d7fa640 1 -- 172.21.15.3:0/898888825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700052a0 con 0x560d88e71980 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- 172.21.15.3:0/898888825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 msgr2=0x560d88e6c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0e7001bc90 tx=0x7f0e7000b400 comp rx=0 tx=0).stop 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- 172.21.15.3:0/898888825 shutdown_connections 2024-09-12T02:23:42.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e7c240 0x560d88e842b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71980 0x560d88e6c640 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 --2- 172.21.15.3:0/898888825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e71060 0x560d88e73ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- 172.21.15.3:0/898888825 >> 172.21.15.3:0/898888825 conn(0x560d88e6d5f0 msgr2=0x560d88e6bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- 172.21.15.3:0/898888825 shutdown_connections 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- 172.21.15.3:0/898888825 wait complete. 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.542+0000 7f0e90524740 1 -- start start 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 Processor -- start 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71980 0x560d88e753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881aeb00 con 0x560d88e71060 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881ae430 con 0x560d88e7c240 2024-09-12T02:23:42.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e881ab670 con 0x560d88e71980 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e90524740 10 client.0 _ll_register_callbacks cb 0x7f0e70004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60312/0 (socket says 172.21.15.3:60312) 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 -- 172.21.15.3:0/4171277786 learned_addr learned my addr 172.21.15.3:0/4171277786 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71980 0x560d88e753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e87fff640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e75910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 -- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71980 msgr2=0x560d88e753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71980 0x560d88e753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 -- 172.21.15.3:0/4171277786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 msgr2=0x560d88e75910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e75910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e877fe640 1 -- 172.21.15.3:0/4171277786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d88e75e50 con 0x560d88e71060 2024-09-12T02:23:42.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.543+0000 7f0e87fff640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e877fe640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009a50 tx=0x7f0e780045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e975b94cbd6008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78049040 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0e7803b460 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e70011440 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0e780bb3a0 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e90524740 10 client.16458 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560d88efb4e0 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70011440 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.544+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f0e780eccb0 con 0x560d88e71060 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 10 client.16458 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 10 client.16458 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 20 client.16458 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:42.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88efba60 0x560d88efde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 4 client.16458 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "704975bd", "release": true, "await": true}] 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "set_id": "704975bd", "release": true, "await": true}) -- 0x560d88dde0a0 con 0x560d88efba60 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88efba60 0x560d88efde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e7ffff640 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88efba60 0x560d88efde50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185d01e5a2db9921 server_cookie=efee1890b168d383 in_seq=0 out_seq=0 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.545+0000 7f0e66ffd640 10 client.16458 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.565+0000 7f0e66ffd640 1 -- 172.21.15.3:0/4171277786 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d88dde0a0 con 0x560d88efba60 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.565+0000 7f0e66ffd640 10 client.16458 handle_command_reply: tid=0 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.565+0000 7f0e90524740 1 client.16458 shutdown 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.566+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 msgr2=0x560d88e76960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.566+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009a50 tx=0x7f0e780045f0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.573+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 shutdown_connections 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.573+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560d88efba60 0x560d88efde50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.573+0000 7f0e66ffd640 0 client.16458 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d88e7c240 0x560d88e75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560d88e71980 0x560d88e753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 --2- 172.21.15.3:0/4171277786 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560d88e71060 0x560d88e76960 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 >> 172.21.15.3:0/4171277786 conn(0x560d88e6d5f0 msgr2=0x560d88e6c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 shutdown_connections 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 1 -- 172.21.15.3:0/4171277786 wait complete. 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.574+0000 7f0e90524740 20 client.16458 trim_cache size 0 max 16384 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.576+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 msgr2=0x7f0e740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 0x7f0e740a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009fd0 tx=0x7f0e80004a40 comp rx=0 tx=0).stop 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 msgr2=0x7f0e88198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009230 tx=0x7f0e78004270 comp rx=0 tx=0).stop 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 shutdown_connections 2024-09-12T02:23:42.496 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0e740a2b80 0x7f0e740a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e88106a80 0x7f0e8819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e88101420 0x7f0e881989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 --2- 172.21.15.3:0/1516072974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e88100a80 0x7f0e88198480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.577+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 >> 172.21.15.3:0/1516072974 conn(0x7f0e880fc7a0 msgr2=0x7f0e880fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.578+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 shutdown_connections 2024-09-12T02:23:42.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:23:41.578+0000 7f0e8e14b640 1 -- 172.21.15.3:0/1516072974 wait complete. 2024-09-12T02:23:42.497 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:42 smithi003 ceph-mon[31476]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 19 MiB/s wr, 3.88k op/s 2024-09-12T02:23:42.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:42 smithi026 ceph-mon[35496]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 19 MiB/s wr, 3.88k op/s 2024-09-12T02:23:42.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:42 smithi092 ceph-mon[36089]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 19 MiB/s wr, 3.88k op/s 2024-09-12T02:23:44.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:44 smithi026 ceph-mon[35496]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 3.24k op/s 2024-09-12T02:23:44.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:44 smithi003 ceph-mon[31476]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 3.24k op/s 2024-09-12T02:23:44.540 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:44 smithi092 ceph-mon[36089]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 3.24k op/s 2024-09-12T02:23:46.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:46 smithi026 ceph-mon[35496]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 MiB/s wr, 2.57k op/s 2024-09-12T02:23:46.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:46 smithi003 ceph-mon[31476]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 MiB/s wr, 2.57k op/s 2024-09-12T02:23:46.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:46 smithi092 ceph-mon[36089]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 13 MiB/s wr, 2.57k op/s 2024-09-12T02:23:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.688+0000 7f47950d0640 1 -- 172.21.15.3:0/470490347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4790101510 msgr2=0x7f4790105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.688+0000 7f47950d0640 1 --2- 172.21.15.3:0/470490347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4790101510 0x7f4790105db0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4788009ca0 tx=0x7f478802a8d0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.689+0000 7f47950d0640 1 -- 172.21.15.3:0/470490347 shutdown_connections 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.689+0000 7f47950d0640 1 --2- 172.21.15.3:0/470490347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4790101510 0x7f4790105db0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.689+0000 7f47950d0640 1 --2- 172.21.15.3:0/470490347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790100b50 0x7f4790100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.689+0000 7f47950d0640 1 --2- 172.21.15.3:0/470490347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f47900ffdf0 0x7f47901001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.689+0000 7f47950d0640 1 -- 172.21.15.3:0/470490347 >> 172.21.15.3:0/470490347 conn(0x7f47900fb930 msgr2=0x7f47900fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.690+0000 7f47950d0640 1 -- 172.21.15.3:0/470490347 shutdown_connections 2024-09-12T02:23:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.692+0000 7f47950d0640 1 -- 172.21.15.3:0/470490347 wait complete. 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 Processor -- start 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 -- start start 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47900ffdf0 0x7f479010c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 0x7f479010c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 0x7f47901ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4790113e90 con 0x7f4790100b50 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f4790113d10 con 0x7f47900ffdf0 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.693+0000 7f47950d0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4790114010 con 0x7f4790101510 2024-09-12T02:23:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.694+0000 7f478f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 0x7f47901ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 0x7f479010c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478e59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 0x7f479010c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60320/0 (socket says 172.21.15.3:60320) 2024-09-12T02:23:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478e59c640 1 -- 172.21.15.3:0/3556476607 learned_addr learned my addr 172.21.15.3:0/3556476607 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478f59e640 1 -- 172.21.15.3:0/3556476607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47900ffdf0 msgr2=0x7f479010c070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:23:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478f59e640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47900ffdf0 0x7f479010c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478f59e640 1 -- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 msgr2=0x7f479010c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478f59e640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 0x7f479010c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.695+0000 7f478f59e640 1 -- 172.21.15.3:0/3556476607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479010f730 con 0x7f4790101510 2024-09-12T02:23:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.698+0000 7f478f59e640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 0x7f47901ae5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f478802aec0 tx=0x7f4788003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.700+0000 7f4777fff640 1 -- 172.21.15.3:0/3556476607 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004450 con 0x7f4790101510 2024-09-12T02:23:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.700+0000 7f4777fff640 1 -- 172.21.15.3:0/3556476607 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788040070 con 0x7f4790101510 2024-09-12T02:23:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.700+0000 7f4777fff640 1 -- 172.21.15.3:0/3556476607 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788030830 con 0x7f4790101510 2024-09-12T02:23:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.702+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479006bc10 con 0x7f4790101510 2024-09-12T02:23:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.702+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479006c0e0 con 0x7f4790101510 2024-09-12T02:23:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.710+0000 7f4777fff640 1 -- 172.21.15.3:0/3556476607 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f47880309d0 con 0x7f4790101510 2024-09-12T02:23:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.710+0000 7f4797e6d740 1 Processor -- start 2024-09-12T02:23:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.711+0000 7f4777fff640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 0x7f47780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.711+0000 7f4777fff640 1 -- 172.21.15.3:0/3556476607 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f47880f07e0 con 0x7f4790101510 2024-09-12T02:23:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 -- start start 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483cd9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483cdeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564483c3d2f0 con 0x564483cdf370 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564483be5270 con 0x564483cd99a0 2024-09-12T02:23:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.712+0000 7f4797e6d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564483c392b0 con 0x564483cda2e0 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60334/0 (socket says 172.21.15.3:60334) 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 -- 172.21.15.3:0/1208118473 learned_addr learned my addr 172.21.15.3:0/1208118473 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 -- 172.21.15.3:0/1208118473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 msgr2=0x564483cdeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483cdeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 -- 172.21.15.3:0/1208118473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 msgr2=0x564483cd9da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:23:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483cd9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.713+0000 7f478f59e640 1 -- 172.21.15.3:0/1208118473 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564483be4340 con 0x564483cdf370 2024-09-12T02:23:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.714+0000 7f478f59e640 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47880b93a0 tx=0x7f478802a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2ef098deb089e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.716+0000 7f47757fa640 1 -- 172.21.15.3:0/1208118473 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880bb110 con 0x564483cdf370 2024-09-12T02:23:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.716+0000 7f47757fa640 1 -- 172.21.15.3:0/1208118473 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788036030 con 0x564483cdf370 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.718+0000 7f478e59c640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 0x7f47780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.718+0000 7f4797e6d740 1 -- 172.21.15.3:0/1208118473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 msgr2=0x564483ce1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.718+0000 7f4797e6d740 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47880b93a0 tx=0x7f478802a8d0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.718+0000 7f4797e6d740 1 -- 172.21.15.3:0/1208118473 shutdown_connections 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.718+0000 7f4797e6d740 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cdf370 0x564483ce1830 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.719+0000 7f4797e6d740 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483cdeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.719+0000 7f4797e6d740 1 --2- 172.21.15.3:0/1208118473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483cd9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.719+0000 7f4797e6d740 1 -- 172.21.15.3:0/1208118473 >> 172.21.15.3:0/1208118473 conn(0x564483cd5010 msgr2=0x564483cd5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.728+0000 7f4797e6d740 1 -- 172.21.15.3:0/1208118473 shutdown_connections 2024-09-12T02:23:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.729+0000 7f4797e6d740 1 -- 172.21.15.3:0/1208118473 wait complete. 2024-09-12T02:23:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.730+0000 7f4797e6d740 1 -- start start 2024-09-12T02:23:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.730+0000 7f4797e6d740 1 Processor -- start 2024-09-12T02:23:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cda2e0 0x564483ce41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ceb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.736 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x564483ce21c0 con 0x564483cda2e0 2024-09-12T02:23:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x564483bc6290 con 0x564483cd99a0 2024-09-12T02:23:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564483ce2340 con 0x564483cdf370 2024-09-12T02:23:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.735+0000 7f4797e6d740 10 client.0 _ll_register_callbacks cb 0x564483bf7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.737+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:59170/0 (socket says 172.21.15.3:59170) 2024-09-12T02:23:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 -- 172.21.15.3:0/3236912029 learned_addr learned my addr 172.21.15.3:0/3236912029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478f59e640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ceb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 -- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 msgr2=0x564483ceb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ceb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 -- 172.21.15.3:0/3236912029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cda2e0 msgr2=0x564483ce41a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cda2e0 0x564483ce41a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.738+0000 7f478ed9d640 1 -- 172.21.15.3:0/3236912029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564483950210 con 0x564483cd99a0 2024-09-12T02:23:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f478f59e640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:23:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f478ed9d640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4784009dd0 tx=0x7f478400b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10396a8b121e3f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f4772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784015020 con 0x564483cd99a0 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564483c4aa20 con 0x564483cd99a0 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.739+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4784011420 con 0x564483cd99a0 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.740+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4784011660 con 0x564483cd99a0 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.740+0000 7f4797e6d740 10 client.25880 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.740+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564483a1c620 con 0x564483cd99a0 2024-09-12T02:23:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.740+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564483c4aa20 con 0x564483cd99a0 2024-09-12T02:23:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.740+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f4784019b20 con 0x564483cd99a0 2024-09-12T02:23:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 10 client.25880 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 20 client.25880 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483cefa00 0x564483cf1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 4 client.25880 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564483c44d70 con 0x564483cefa00 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.741+0000 7f478e59c640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 0x7f47780a5260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564483cdad10 tx=0x7f4780009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.742+0000 7f478e59c640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483cefa00 0x564483cf1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.743+0000 7f478e59c640 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483cefa00 0x564483cf1ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da95b7dbcc75318 server_cookie=a1dc7bf89d2eb28d in_seq=0 out_seq=0 2024-09-12T02:23:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.743+0000 7f4772ff5640 10 client.25880 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.745+0000 7f4772ff5640 1 -- 172.21.15.3:0/3236912029 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564483c44d70 con 0x564483cefa00 2024-09-12T02:23:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.745+0000 7f4772ff5640 10 client.25880 handle_command_reply: tid=0 2024-09-12T02:23:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.745+0000 7f4797e6d740 1 client.25880 shutdown 2024-09-12T02:23:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.750+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 msgr2=0x564483ce3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.750+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4784009dd0 tx=0x7f478400b4b0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 shutdown_connections 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483cefa00 0x564483cf1ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cda2e0 0x564483ce41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3236912029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cd99a0 0x564483ce3c60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 >> 172.21.15.3:0/3236912029 conn(0x564483cd5010 msgr2=0x564483cd6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.751+0000 7f4772ff5640 0 client.25880 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.754+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 shutdown_connections 2024-09-12T02:23:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.754+0000 7f4797e6d740 1 -- 172.21.15.3:0/3236912029 wait complete. 2024-09-12T02:23:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.755+0000 7f4797e6d740 20 client.25880 trim_cache size 0 max 16384 2024-09-12T02:23:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.797+0000 7f4797e6d740 1 Processor -- start 2024-09-12T02:23:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.798+0000 7f4797e6d740 1 -- start start 2024-09-12T02:23:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.798+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483cd5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483ceb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483cd5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483cd5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60338/0 (socket says 172.21.15.3:60338) 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f478ed9d640 1 -- 172.21.15.3:0/3269028194 learned_addr learned my addr 172.21.15.3:0/3269028194 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 0x564483ce4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47901c06e0 con 0x564483cd99a0 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.799+0000 7f478f59e640 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 0x564483ce4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.800+0000 7f478e59c640 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483ceb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.800+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47901c2220 con 0x564483cdf370 2024-09-12T02:23:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.800+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f47901c0db0 con 0x564483cda2e0 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.801+0000 7f478f59e640 1 -- 172.21.15.3:0/3269028194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 msgr2=0x564483ceb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.801+0000 7f478f59e640 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483ceb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.801+0000 7f478f59e640 1 -- 172.21.15.3:0/3269028194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 msgr2=0x564483cd5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.801+0000 7f478f59e640 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483cd5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.801+0000 7f478f59e640 1 -- 172.21.15.3:0/3269028194 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564483cd4e30 con 0x564483cdf370 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.802+0000 7f478f59e640 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 0x564483ce4120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4788031ab0 tx=0x7f47880bcc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5347da12575ddcdf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.802+0000 7f4774ff9640 1 -- 172.21.15.3:0/3269028194 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880bb110 con 0x564483cdf370 2024-09-12T02:23:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.802+0000 7f4774ff9640 1 -- 172.21.15.3:0/3269028194 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788036040 con 0x564483cdf370 2024-09-12T02:23:47.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.802+0000 7f4774ff9640 1 -- 172.21.15.3:0/3269028194 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880bb2d0 con 0x564483cdf370 2024-09-12T02:23:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.804+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 msgr2=0x564483ce4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.804+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 0x564483ce4120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4788031ab0 tx=0x7f47880bcc10 comp rx=0 tx=0).stop 2024-09-12T02:23:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.805+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 shutdown_connections 2024-09-12T02:23:47.805 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.805+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cdf370 0x564483ce4120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.805+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cda2e0 0x564483ceb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.805+0000 7f4797e6d740 1 --2- 172.21.15.3:0/3269028194 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483cd5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.805+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 >> 172.21.15.3:0/3269028194 conn(0x564483cd5010 msgr2=0x564483ce3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.808+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 shutdown_connections 2024-09-12T02:23:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.811+0000 7f4797e6d740 1 -- 172.21.15.3:0/3269028194 wait complete. 2024-09-12T02:23:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.812+0000 7f4797e6d740 1 -- start start 2024-09-12T02:23:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.812+0000 7f4797e6d740 1 Processor -- start 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cda2e0 0x564483ce8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ce97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f47901c1b50 con 0x564483cd99a0 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f47901c1480 con 0x564483cda2e0 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4790072ff0 con 0x564483cdf370 2024-09-12T02:23:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.815+0000 7f4797e6d740 10 client.0 _ll_register_callbacks cb 0x7f4784006e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:23:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.816+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.816+0000 7f478ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60350/0 (socket says 172.21.15.3:60350) 2024-09-12T02:23:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.816+0000 7f478ed9d640 1 -- 172.21.15.3:0/983413851 learned_addr learned my addr 172.21.15.3:0/983413851 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:23:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.816+0000 7f478f59e640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ce97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.817+0000 7f478ed9d640 1 -- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 msgr2=0x564483ce97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:23:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.817+0000 7f478ed9d640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ce97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.817+0000 7f478ed9d640 1 -- 172.21.15.3:0/983413851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cda2e0 msgr2=0x564483ce8790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:23:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.817+0000 7f478ed9d640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cda2e0 0x564483ce8790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.817+0000 7f478ed9d640 1 -- 172.21.15.3:0/983413851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564483ce9db0 con 0x564483cd99a0 2024-09-12T02:23:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.821+0000 7f478ed9d640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540665938fd533ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:23:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.822+0000 7f4772ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:23:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.823+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784027070 con 0x564483cd99a0 2024-09-12T02:23:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.823+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4788036040 con 0x564483cd99a0 2024-09-12T02:23:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.823+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f4784011800 con 0x564483cd99a0 2024-09-12T02:23:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.825+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4784030050 con 0x564483cd99a0 2024-09-12T02:23:47.828 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.828+0000 7f4797e6d740 10 client.16470 fetch_fsmap learned FSMap version 37 2024-09-12T02:23:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.828+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564483ceb190 con 0x564483cd99a0 2024-09-12T02:23:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.830+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788036040 con 0x564483cd99a0 2024-09-12T02:23:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.831+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f47840193a0 con 0x564483cd99a0 2024-09-12T02:23:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 10 client.16470 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:23:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:23:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 20 client.16470 populate_metadata read hostname 'smithi003' 2024-09-12T02:23:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483d5ab80 0x564483d5cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:23:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 4 client.16470 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:23:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.832+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564483bc1900 con 0x564483d5ab80 2024-09-12T02:23:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.838+0000 7f478e59c640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483d5ab80 0x564483d5cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:23:47.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.846+0000 7f478e59c640 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483d5ab80 0x564483d5cfa0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d4dfab8dfd88ac8 server_cookie=62ffdfcf8d7f8e46 in_seq=0 out_seq=0 2024-09-12T02:23:47.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:47.849+0000 7f4772ff5640 10 client.16470 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:48.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:48 smithi026 ceph-mon[35496]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 3.96k op/s 2024-09-12T02:23:48.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:48 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:23:48.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:48 smithi003 ceph-mon[31476]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 3.96k op/s 2024-09-12T02:23:48.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:48 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:23:48.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:48 smithi092 ceph-mon[36089]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 3.96k op/s 2024-09-12T02:23:48.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:48 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:23:50.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:50 smithi026 ceph-mon[35496]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 27 MiB/s wr, 4.42k op/s 2024-09-12T02:23:50.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:50 smithi003 ceph-mon[31476]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 27 MiB/s wr, 4.42k op/s 2024-09-12T02:23:50.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:50 smithi092 ceph-mon[36089]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 27 MiB/s wr, 4.42k op/s 2024-09-12T02:23:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4772ff5640 1 -- 172.21.15.3:0/983413851 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564483bc1900 con 0x564483d5ab80 2024-09-12T02:23:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4772ff5640 10 client.16470 handle_command_reply: tid=0 2024-09-12T02:23:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 client.16470 shutdown 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 msgr2=0x564483ce8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4784006fd0 tx=0x7f4784004270 comp rx=0 tx=0).stop 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 shutdown_connections 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x564483d5ab80 0x564483d5cfa0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564483cdf370 0x564483ce97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564483cda2e0 0x564483ce8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 --2- 172.21.15.3:0/983413851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x564483cd99a0 0x564483ce8250 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 >> 172.21.15.3:0/983413851 conn(0x564483cd5010 msgr2=0x564483cd39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.069+0000 7f4772ff5640 0 client.16470 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.070+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 shutdown_connections 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.070+0000 7f4797e6d740 1 -- 172.21.15.3:0/983413851 wait complete. 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.070+0000 7f4797e6d740 20 client.16470 trim_cache size 0 max 16384 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:23:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 msgr2=0x7f47780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 0x7f47780a5260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x564483cdad10 tx=0x7f4780009620 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 msgr2=0x7f47901ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 0x7f47901ae5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f478802aec0 tx=0x7f4788003f10 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 shutdown_connections 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f47780a2da0 0x7f47780a5260 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4790101510 0x7f47901ae5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4790100b50 0x7f479010c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.071+0000 7f47950d0640 1 --2- 172.21.15.3:0/3556476607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47900ffdf0 0x7f479010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.072+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 >> 172.21.15.3:0/3556476607 conn(0x7f47900fb930 msgr2=0x7f4790101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:23:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.072+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 shutdown_connections 2024-09-12T02:23:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:23:51.072+0000 7f47950d0640 1 -- 172.21.15.3:0/3556476607 wait complete. 2024-09-12T02:23:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:52 smithi026 ceph-mon[35496]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 44 MiB/s wr, 6.89k op/s 2024-09-12T02:23:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:23:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:23:52.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:23:52.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:52 smithi003 ceph-mon[31476]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 44 MiB/s wr, 6.89k op/s 2024-09-12T02:23:52.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:23:52.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:23:52.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:23:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:52 smithi092 ceph-mon[36089]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 44 MiB/s wr, 6.89k op/s 2024-09-12T02:23:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:23:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:23:52.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:23:52.798 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '704975bd', total seconds elapsed: 96.3 2024-09-12T02:23:54.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:54 smithi026 ceph-mon[35496]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 53 MiB/s wr, 8.34k op/s 2024-09-12T02:23:54.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:54 smithi003 ceph-mon[31476]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 53 MiB/s wr, 8.34k op/s 2024-09-12T02:23:54.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:54 smithi092 ceph-mon[36089]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 53 MiB/s wr, 8.34k op/s 2024-09-12T02:23:56.221 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:56 smithi003 ceph-mon[31476]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 59 MiB/s wr, 9.25k op/s 2024-09-12T02:23:56.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:56 smithi026 ceph-mon[35496]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 59 MiB/s wr, 9.25k op/s 2024-09-12T02:23:56.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:56 smithi092 ceph-mon[36089]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 59 MiB/s wr, 9.25k op/s 2024-09-12T02:23:58.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:23:58 smithi026 ceph-mon[35496]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 82 MiB/s wr, 12.90k op/s 2024-09-12T02:23:58.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:23:58 smithi003 ceph-mon[31476]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 82 MiB/s wr, 12.90k op/s 2024-09-12T02:23:58.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:23:58 smithi092 ceph-mon[36089]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 82 MiB/s wr, 12.90k op/s 2024-09-12T02:24:00.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:00 smithi026 ceph-mon[35496]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 76 MiB/s wr, 11.95k op/s 2024-09-12T02:24:00.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:00 smithi003 ceph-mon[31476]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 76 MiB/s wr, 11.95k op/s 2024-09-12T02:24:00.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:00 smithi092 ceph-mon[36089]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 76 MiB/s wr, 11.95k op/s 2024-09-12T02:24:02.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:02 smithi026 ceph-mon[35496]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 87 MiB/s wr, 13.45k op/s 2024-09-12T02:24:02.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:02 smithi003 ceph-mon[31476]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 87 MiB/s wr, 13.45k op/s 2024-09-12T02:24:02.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:02 smithi092 ceph-mon[36089]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 87 MiB/s wr, 13.45k op/s 2024-09-12T02:24:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.237+0000 7f651599f640 1 -- 172.21.15.3:0/339978662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 msgr2=0x7f651006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.237+0000 7f651599f640 1 --2- 172.21.15.3:0/339978662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f651006fd90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6500009ca0 tx=0x7f650002a8f0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.240+0000 7f651599f640 1 -- 172.21.15.3:0/339978662 shutdown_connections 2024-09-12T02:24:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.240+0000 7f651599f640 1 --2- 172.21.15.3:0/339978662 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f651006fd90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.240+0000 7f651599f640 1 --2- 172.21.15.3:0/339978662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651006eb50 0x7f651006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.240+0000 7f651599f640 1 --2- 172.21.15.3:0/339978662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f651006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.240+0000 7f651599f640 1 -- 172.21.15.3:0/339978662 >> 172.21.15.3:0/339978662 conn(0x7f651006c2b0 msgr2=0x7f651006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.244+0000 7f651599f640 1 -- 172.21.15.3:0/339978662 shutdown_connections 2024-09-12T02:24:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.247+0000 7f651599f640 1 -- 172.21.15.3:0/339978662 wait complete. 2024-09-12T02:24:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.248+0000 7f651599f640 1 Processor -- start 2024-09-12T02:24:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 -- start start 2024-09-12T02:24:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f6510119eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651006eb50 0x7f651011a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f65101184f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f651010cc60 con 0x7f651006f4f0 2024-09-12T02:24:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f651010cae0 con 0x7f651006eb50 2024-09-12T02:24:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.249+0000 7f651599f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f651010cde0 con 0x7f651006d910 2024-09-12T02:24:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.254+0000 7f650effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f6510119eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.254+0000 7f650effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f6510119eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:58618/0 (socket says 172.21.15.3:58618) 2024-09-12T02:24:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.254+0000 7f650effd640 1 -- 172.21.15.3:0/1994444646 learned_addr learned my addr 172.21.15.3:0/1994444646 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.256+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f65101184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.257+0000 7f650f7fe640 1 -- 172.21.15.3:0/1994444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 msgr2=0x7f6510119eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.257+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f6510119eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.257+0000 7f650f7fe640 1 -- 172.21.15.3:0/1994444646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651006eb50 msgr2=0x7f651011a3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:24:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.257+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651006eb50 0x7f651011a3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.257+0000 7f650f7fe640 1 -- 172.21.15.3:0/1994444646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6510118d60 con 0x7f651006f4f0 2024-09-12T02:24:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f65101184f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6500031af0 tx=0x7f6500003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f651499d640 1 -- 172.21.15.3:0/1994444646 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500003e10 con 0x7f651006f4f0 2024-09-12T02:24:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6510118ff0 con 0x7f651006f4f0 2024-09-12T02:24:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65101b7bb0 con 0x7f651006f4f0 2024-09-12T02:24:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f651499d640 1 -- 172.21.15.3:0/1994444646 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f650002d070 con 0x7f651006f4f0 2024-09-12T02:24:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.259+0000 7f651499d640 1 -- 172.21.15.3:0/1994444646 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650002fc20 con 0x7f651006f4f0 2024-09-12T02:24:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.263+0000 7f651499d640 1 -- 172.21.15.3:0/1994444646 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f65000305f0 con 0x7f651006f4f0 2024-09-12T02:24:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.266+0000 7f651499d640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 0x7f64fc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.267+0000 7f651499d640 1 -- 172.21.15.3:0/1994444646 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f65000f4060 con 0x7f651006f4f0 2024-09-12T02:24:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 Processor -- start 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 -- start start 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814af9a0 0x5567814afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567814132f0 con 0x5567814b02e0 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5567813bb270 con 0x5567814af9a0 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f6518815740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55678140f2b0 con 0x5567814b5370 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:58620/0 (socket says 172.21.15.3:58620) 2024-09-12T02:24:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.268+0000 7f650f7fe640 1 -- 172.21.15.3:0/1112824141 learned_addr learned my addr 172.21.15.3:0/1112824141 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 -- 172.21.15.3:0/1112824141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814af9a0 msgr2=0x5567814afda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814af9a0 0x5567814afda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 -- 172.21.15.3:0/1112824141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 msgr2=0x5567814b4da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 -- 172.21.15.3:0/1112824141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567813ba340 con 0x5567814b5370 2024-09-12T02:24:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f65000b8af0 tx=0x7f65000bc520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721b3d45ec3c4586 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f6505ffb640 1 -- 172.21.15.3:0/1112824141 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000efe60 con 0x5567814b5370 2024-09-12T02:24:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f6505ffb640 1 -- 172.21.15.3:0/1112824141 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6500036040 con 0x5567814b5370 2024-09-12T02:24:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f6505ffb640 1 -- 172.21.15.3:0/1112824141 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650002fdc0 con 0x5567814b5370 2024-09-12T02:24:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.269+0000 7f650effd640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 0x7f64fc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.270+0000 7f6518815740 1 -- 172.21.15.3:0/1112824141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 msgr2=0x5567814b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f65000b8af0 tx=0x7f65000bc520 comp rx=0 tx=0).stop 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 -- 172.21.15.3:0/1112824141 shutdown_connections 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814b7830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 --2- 172.21.15.3:0/1112824141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814af9a0 0x5567814afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.272+0000 7f6518815740 1 -- 172.21.15.3:0/1112824141 >> 172.21.15.3:0/1112824141 conn(0x5567814ab010 msgr2=0x5567814ab410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.274+0000 7f6518815740 1 -- 172.21.15.3:0/1112824141 shutdown_connections 2024-09-12T02:24:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.274+0000 7f650effd640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 0x7f64fc0a5100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6508006fd0 tx=0x7f6508008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.276+0000 7f6518815740 1 -- 172.21.15.3:0/1112824141 wait complete. 2024-09-12T02:24:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.276+0000 7f6518815740 1 -- start start 2024-09-12T02:24:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.276+0000 7f6518815740 1 Processor -- start 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814af9a0 0x5567814b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567814b8100 con 0x5567814b02e0 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55678139c290 con 0x5567814b5370 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.279+0000 7f6518815740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5567814b8280 con 0x5567814af9a0 2024-09-12T02:24:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.280+0000 7f6518815740 10 client.0 _ll_register_callbacks cb 0x5567813cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.280+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.280+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:56056/0 (socket says 172.21.15.3:56056) 2024-09-12T02:24:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.280+0000 7f650f7fe640 1 -- 172.21.15.3:0/1731447758 learned_addr learned my addr 172.21.15.3:0/1731447758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.280+0000 7f650effd640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814af9a0 0x5567814b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650e7fc640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 -- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814af9a0 msgr2=0x5567814b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814af9a0 0x5567814b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 -- 172.21.15.3:0/1731447758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 msgr2=0x5567814b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 -- 172.21.15.3:0/1731447758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556781126210 con 0x5567814b5370 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650e7fc640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:24:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.281+0000 7f650f7fe640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6500006010 tx=0x7f65000c0a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e26af56e2ea9181 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.284+0000 7f64eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:24:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.284+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000d9060 con 0x5567814b5370 2024-09-12T02:24:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.284+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6500036040 con 0x5567814b5370 2024-09-12T02:24:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.284+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f650002fdc0 con 0x5567814b5370 2024-09-12T02:24:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.287+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f65000bcd90 con 0x5567814b5370 2024-09-12T02:24:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.287+0000 7f6518815740 10 client.25886 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.287+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5567811f2620 con 0x5567814b5370 2024-09-12T02:24:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.289+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6500036040 con 0x5567814b5370 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.289+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f65000c4e70 con 0x5567814b5370 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 10 client.25886 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 20 client.25886 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567814c5550 0x5567814c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 4 client.25886 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.290+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556781420a20 con 0x5567814c5550 2024-09-12T02:24:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.291+0000 7f650effd640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567814c5550 0x5567814c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.291+0000 7f64eeffd640 10 client.25886 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.292+0000 7f650effd640 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567814c5550 0x5567814c7970 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a08a7ccf45ec49d server_cookie=df31c85afdedf0bc in_seq=0 out_seq=0 2024-09-12T02:24:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.294+0000 7f64eeffd640 1 -- 172.21.15.3:0/1731447758 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556781420a20 con 0x5567814c5550 2024-09-12T02:24:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.294+0000 7f64eeffd640 10 client.25886 handle_command_reply: tid=0 2024-09-12T02:24:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.294+0000 7f6518815740 1 client.25886 shutdown 2024-09-12T02:24:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.296+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 msgr2=0x5567814c0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.296+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6500006010 tx=0x7f65000c0a40 comp rx=0 tx=0).stop 2024-09-12T02:24:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 shutdown_connections 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5567814c5550 0x5567814c7970 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814c0e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814b02e0 0x5567814b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 --2- 172.21.15.3:0/1731447758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814af9a0 0x5567814b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.298+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 >> 172.21.15.3:0/1731447758 conn(0x5567814ab010 msgr2=0x5567814ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.300+0000 7f64eeffd640 0 client.25886 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.301+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 shutdown_connections 2024-09-12T02:24:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.302+0000 7f6518815740 1 -- 172.21.15.3:0/1731447758 wait complete. 2024-09-12T02:24:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.302+0000 7f6518815740 20 client.25886 trim_cache size 0 max 16384 2024-09-12T02:24:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.340+0000 7f6518815740 1 Processor -- start 2024-09-12T02:24:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.345+0000 7f6518815740 1 -- start start 2024-09-12T02:24:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.345+0000 7f6518815740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.345+0000 7f6518815740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b02e0 0x5567814b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.345+0000 7f6518815740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f6518815740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000e5c0 con 0x5567814af9a0 2024-09-12T02:24:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f650f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:58644/0 (socket says 172.21.15.3:58644) 2024-09-12T02:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f650f7fe640 1 -- 172.21.15.3:0/3154617856 learned_addr learned my addr 172.21.15.3:0/3154617856 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f650e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b02e0 0x5567814b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0010100 con 0x5567814b02e0 2024-09-12T02:24:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.346+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000ec90 con 0x5567814b5370 2024-09-12T02:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.348+0000 7f650f7fe640 1 -- 172.21.15.3:0/3154617856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b02e0 msgr2=0x5567814b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.348+0000 7f650f7fe640 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b02e0 0x5567814b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.348+0000 7f650f7fe640 1 -- 172.21.15.3:0/3154617856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 msgr2=0x5567814b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.348+0000 7f650f7fe640 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.348+0000 7f650f7fe640 1 -- 172.21.15.3:0/3154617856 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567814aae30 con 0x5567814b5370 2024-09-12T02:24:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.350+0000 7f650f7fe640 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6500003fd0 tx=0x7f65000bc520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b84596d0e06b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.354+0000 7f65057fa640 1 -- 172.21.15.3:0/3154617856 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000d9070 con 0x5567814b5370 2024-09-12T02:24:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.354+0000 7f65057fa640 1 -- 172.21.15.3:0/3154617856 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65000f1070 con 0x5567814b5370 2024-09-12T02:24:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.354+0000 7f65057fa640 1 -- 172.21.15.3:0/3154617856 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65000bc8f0 con 0x5567814b5370 2024-09-12T02:24:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.355+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 msgr2=0x5567814c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.355+0000 7f6518815740 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6500003fd0 tx=0x7f65000bc520 comp rx=0 tx=0).stop 2024-09-12T02:24:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.357+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 shutdown_connections 2024-09-12T02:24:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.357+0000 7f6518815740 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b5370 0x5567814c0dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.357+0000 7f6518815740 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b02e0 0x5567814b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.357+0000 7f6518815740 1 --2- 172.21.15.3:0/3154617856 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.357+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 >> 172.21.15.3:0/3154617856 conn(0x5567814ab010 msgr2=0x5567814b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.362+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 shutdown_connections 2024-09-12T02:24:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.364+0000 7f6518815740 1 -- 172.21.15.3:0/3154617856 wait complete. 2024-09-12T02:24:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.364+0000 7f6518815740 1 -- start start 2024-09-12T02:24:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.364+0000 7f6518815740 1 Processor -- start 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.365+0000 7f6518815740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.365+0000 7f6518815740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 0x5567814bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.365+0000 7f6518815740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f6518815740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000fa30 con 0x5567814af9a0 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f6518815740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000f360 con 0x5567814b5370 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f650e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 0x5567814bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f6518815740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f64f000c5a0 con 0x5567814b02e0 2024-09-12T02:24:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f650e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 0x5567814bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:58646/0 (socket says 172.21.15.3:58646) 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f650e7fc640 1 -- 172.21.15.3:0/420579692 learned_addr learned my addr 172.21.15.3:0/420579692 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f6518815740 10 client.0 _ll_register_callbacks cb 0x7f65000b8930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f650f7fe640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.366+0000 7f650effd640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.367+0000 7f650effd640 1 -- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 msgr2=0x5567814bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.367+0000 7f650effd640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 0x5567814bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.367+0000 7f650effd640 1 -- 172.21.15.3:0/420579692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 msgr2=0x5567814bdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.367+0000 7f650effd640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814bdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.367+0000 7f650effd640 1 -- 172.21.15.3:0/420579692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55678152e240 con 0x5567814af9a0 2024-09-12T02:24:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.369+0000 7f650effd640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814c0be0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x5567814c81f0 tx=0x7f650800daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42c61212a8bb2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.369+0000 7f64eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:24:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.372+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650800de40 con 0x5567814af9a0 2024-09-12T02:24:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.373+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65000f1070 con 0x5567814af9a0 2024-09-12T02:24:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.373+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f650800f040 con 0x5567814af9a0 2024-09-12T02:24:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.374+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f65080168a0 con 0x5567814af9a0 2024-09-12T02:24:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.375+0000 7f6518815740 10 client.16479 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.375+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55678152f500 con 0x5567814af9a0 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.380+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65000f1070 con 0x5567814af9a0 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.380+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f6508013070 con 0x5567814af9a0 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 10 client.16479 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 20 client.16479 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55678152fe30 0x556781532250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 4 client.16479 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:24:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.381+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55678141ad70 con 0x55678152fe30 2024-09-12T02:24:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.382+0000 7f650e7fc640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55678152fe30 0x556781532250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.385+0000 7f650e7fc640 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55678152fe30 0x556781532250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e485eddc977253c8 server_cookie=918f63a67376765 in_seq=0 out_seq=0 2024-09-12T02:24:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:04.385+0000 7f64eeffd640 10 client.16479 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:04.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:04 smithi026 ceph-mon[35496]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 95 MiB/s wr, 14.54k op/s 2024-09-12T02:24:04.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:04 smithi003 ceph-mon[31476]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 95 MiB/s wr, 14.54k op/s 2024-09-12T02:24:04.538 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:04 smithi092 ceph-mon[36089]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 95 MiB/s wr, 14.54k op/s 2024-09-12T02:24:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.264+0000 7f64eeffd640 1 -- 172.21.15.3:0/420579692 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55678141ad70 con 0x55678152fe30 2024-09-12T02:24:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f64eeffd640 10 client.16479 handle_command_reply: tid=0 2024-09-12T02:24:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 client.16479 shutdown 2024-09-12T02:24:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 msgr2=0x5567814c0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814c0be0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x5567814c81f0 tx=0x7f650800daf0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 shutdown_connections 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55678152fe30 0x556781532250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5567814b5370 0x5567814bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5567814b02e0 0x5567814bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 --2- 172.21.15.3:0/420579692 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567814af9a0 0x5567814c0be0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 >> 172.21.15.3:0/420579692 conn(0x5567814ab010 msgr2=0x5567814b9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.265+0000 7f64eeffd640 0 client.16479 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.266+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 shutdown_connections 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.266+0000 7f6518815740 1 -- 172.21.15.3:0/420579692 wait complete. 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.266+0000 7f6518815740 20 client.16479 trim_cache size 0 max 16384 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:24:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:24:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 msgr2=0x7f64fc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 0x7f64fc0a5100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6508006fd0 tx=0x7f6508008040 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 msgr2=0x7f65101184f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f65101184f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6500031af0 tx=0x7f6500003ba0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 shutdown_connections 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f64fc0a2c40 0x7f64fc0a5100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f651006f4f0 0x7f65101184f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651006eb50 0x7f651011a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 --2- 172.21.15.3:0/1994444646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f651006d910 0x7f6510119eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.267+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 >> 172.21.15.3:0/1994444646 conn(0x7f651006c2b0 msgr2=0x7f651010af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.268+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 shutdown_connections 2024-09-12T02:24:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:05.268+0000 7f651599f640 1 -- 172.21.15.3:0/1994444646 wait complete. 2024-09-12T02:24:06.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:06 smithi026 ceph-mon[35496]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-09-12T02:24:06.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:06 smithi003 ceph-mon[31476]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-09-12T02:24:06.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:06 smithi092 ceph-mon[36089]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 90 MiB/s wr, 13.65k op/s 2024-09-12T02:24:08.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:08 smithi026 ceph-mon[35496]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 121 MiB/s wr, 17.69k op/s 2024-09-12T02:24:08.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:08 smithi003 ceph-mon[31476]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 121 MiB/s wr, 17.69k op/s 2024-09-12T02:24:08.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:08 smithi092 ceph-mon[36089]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 121 MiB/s wr, 17.69k op/s 2024-09-12T02:24:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:10 smithi026 ceph-mon[35496]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 108 MiB/s wr, 15.40k op/s 2024-09-12T02:24:10.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:10 smithi003 ceph-mon[31476]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 108 MiB/s wr, 15.40k op/s 2024-09-12T02:24:10.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:10 smithi092 ceph-mon[36089]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 108 MiB/s wr, 15.40k op/s 2024-09-12T02:24:12.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:12 smithi026 ceph-mon[35496]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 105 MiB/s wr, 14.84k op/s 2024-09-12T02:24:12.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:12 smithi003 ceph-mon[31476]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 105 MiB/s wr, 14.84k op/s 2024-09-12T02:24:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:12 smithi092 ceph-mon[36089]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 38 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 105 MiB/s wr, 14.84k op/s 2024-09-12T02:24:14.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:14 smithi026 ceph-mon[35496]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 125 MiB/s wr, 17.14k op/s 2024-09-12T02:24:14.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:14 smithi003 ceph-mon[31476]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 125 MiB/s wr, 17.14k op/s 2024-09-12T02:24:14.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:14 smithi092 ceph-mon[36089]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 125 MiB/s wr, 17.14k op/s 2024-09-12T02:24:16.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:16 smithi026 ceph-mon[35496]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 108 MiB/s wr, 14.48k op/s 2024-09-12T02:24:16.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:16 smithi003 ceph-mon[31476]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 108 MiB/s wr, 14.48k op/s 2024-09-12T02:24:16.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:16 smithi092 ceph-mon[36089]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 108 MiB/s wr, 14.48k op/s 2024-09-12T02:24:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1436031220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0073790 msgr2=0x7f2fb0112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1436031220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0073790 0x7f2fb0112540 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009ca0 tx=0x7f2fa802a8d0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1436031220 shutdown_connections 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1436031220 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0073790 0x7f2fb0112540 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1436031220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb0073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.395+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1436031220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0109930 0x7f2fb0109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.396+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1436031220 >> 172.21.15.3:0/1436031220 conn(0x7f2fb006ecd0 msgr2=0x7f2fb0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.397+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1436031220 shutdown_connections 2024-09-12T02:24:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.397+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1436031220 wait complete. 2024-09-12T02:24:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.398+0000 7f2fb49c4640 1 Processor -- start 2024-09-12T02:24:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.398+0000 7f2fb49c4640 1 -- start start 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb01a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 0x7f2fb01a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0109930 0x7f2fb01ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010c2e0 con 0x7f2fb0109930 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010c160 con 0x7f2fb0072d10 2024-09-12T02:24:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fb49c4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0113ff0 con 0x7f2fb0073790 2024-09-12T02:24:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 0x7f2fb01a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 0x7f2fb01a7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:52762/0 (socket says 172.21.15.3:52762) 2024-09-12T02:24:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fadd9b640 1 -- 172.21.15.3:0/1964134337 learned_addr learned my addr 172.21.15.3:0/1964134337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fae59c640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb01a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2faed9d640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0109930 0x7f2fb01ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fae59c640 1 -- 172.21.15.3:0/1964134337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 msgr2=0x7f2fb01a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fae59c640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 0x7f2fb01a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.399+0000 7f2fae59c640 1 -- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0109930 msgr2=0x7f2fb01ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fae59c640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0109930 0x7f2fb01ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fae59c640 1 -- 172.21.15.3:0/1964134337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01ae750 con 0x7f2fb0072d10 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fae59c640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb01a6b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c970 tx=0x7f2fa000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fa77fe640 1 -- 172.21.15.3:0/1964134337 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0007bb0 con 0x7f2fb0072d10 2024-09-12T02:24:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fa77fe640 1 -- 172.21.15.3:0/1964134337 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa0007d50 con 0x7f2fb0072d10 2024-09-12T02:24:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01aea40 con 0x7f2fb0072d10 2024-09-12T02:24:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fa77fe640 1 -- 172.21.15.3:0/1964134337 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0018820 con 0x7f2fb0072d10 2024-09-12T02:24:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.400+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01aef90 con 0x7f2fb0072d10 2024-09-12T02:24:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.403+0000 7f2fa77fe640 1 -- 172.21.15.3:0/1964134337 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2fa00189c0 con 0x7f2fb0072d10 2024-09-12T02:24:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.404+0000 7f2fb7841740 1 Processor -- start 2024-09-12T02:24:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 -- start start 2024-09-12T02:24:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fa77fe640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a0993da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fa77fe640 1 -- 172.21.15.3:0/1964134337 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2fa00cf040 con 0x7f2fb0072d10 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2faed9d640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a0998da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a099b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5644a08f72f0 con 0x5644a09939a0 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644a089f270 con 0x5644a0999370 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fb7841740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644a08f32b0 con 0x5644a09942e0 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a0993da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a0993da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37900/0 (socket says 172.21.15.3:37900) 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fae59c640 1 -- 172.21.15.3:0/264705472 learned_addr learned my addr 172.21.15.3:0/264705472 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a0998da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.405+0000 7f2faed9d640 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a099b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 -- 172.21.15.3:0/264705472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 msgr2=0x5644a099b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a099b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 -- 172.21.15.3:0/264705472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 msgr2=0x5644a0993da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a0993da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 -- 172.21.15.3:0/264705472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a089e340 con 0x5644a09942e0 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2faed9d640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005e30 tx=0x7f2fa8031460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a0998da0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f98005fd0 tx=0x7f2f980107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2470c5f93de84da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fa4ff9640 1 -- 172.21.15.3:0/264705472 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98004030 con 0x5644a09942e0 2024-09-12T02:24:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fa4ff9640 1 -- 172.21.15.3:0/264705472 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f980041d0 con 0x5644a09942e0 2024-09-12T02:24:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.406+0000 7f2fa4ff9640 1 -- 172.21.15.3:0/264705472 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980116a0 con 0x5644a09942e0 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 -- 172.21.15.3:0/264705472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 msgr2=0x5644a0998da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a0998da0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f98005fd0 tx=0x7f2f980107a0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 -- 172.21.15.3:0/264705472 shutdown_connections 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a099b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a0998da0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 --2- 172.21.15.3:0/264705472 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a0993da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 -- 172.21.15.3:0/264705472 >> 172.21.15.3:0/264705472 conn(0x5644a098f010 msgr2=0x5644a098f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 -- 172.21.15.3:0/264705472 shutdown_connections 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.407+0000 7f2fb7841740 1 -- 172.21.15.3:0/264705472 wait complete. 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 -- start start 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 Processor -- start 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a099dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09942e0 0x5644a099e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5644a099c340 con 0x5644a09939a0 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5644a0880290 con 0x5644a09942e0 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.408+0000 7f2fb7841740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5644a099c4c0 con 0x5644a0999370 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a099dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:52782/0 (socket says 172.21.15.3:52782) 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 -- 172.21.15.3:0/3606315969 learned_addr learned my addr 172.21.15.3:0/3606315969 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2fb7841740 10 client.0 _ll_register_callbacks cb 0x5644a08b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09942e0 0x5644a099e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 -- 172.21.15.3:0/3606315969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09942e0 msgr2=0x5644a099e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09942e0 0x5644a099e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 -- 172.21.15.3:0/3606315969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 msgr2=0x5644a099dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a099dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.409+0000 7f2faed9d640 1 -- 172.21.15.3:0/3606315969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a060a210 con 0x5644a0999370 2024-09-12T02:24:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.411+0000 7f2faed9d640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009ca0 tx=0x7f2fa802a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a12d52a648678880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.411+0000 7f2f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:24:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.413+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8045040 con 0x5644a0999370 2024-09-12T02:24:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.413+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2fa802f630 con 0x5644a0999370 2024-09-12T02:24:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.414+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f980041d0 con 0x5644a0999370 2024-09-12T02:24:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.415+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2fa8004820 con 0x5644a0999370 2024-09-12T02:24:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.418+0000 7f2fb7841740 10 client.26050 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:18.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.418+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5644a06d6620 con 0x5644a0999370 2024-09-12T02:24:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.421+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f980041d0 con 0x5644a0999370 2024-09-12T02:24:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.421+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f2fa8048030 con 0x5644a0999370 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 10 client.26050 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 20 client.26050 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a09a97d0 0x5644a09abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 4 client.26050 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644a0904a20 con 0x5644a09a97d0 2024-09-12T02:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.422+0000 7f2fae59c640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a09a97d0 0x5644a09abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.423+0000 7f2fae59c640 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a09a97d0 0x5644a09abbc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48b58cf3ebd44fd server_cookie=88d5025a0dc913ea in_seq=0 out_seq=0 2024-09-12T02:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.423+0000 7f2f95ffb640 10 client.26050 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.424+0000 7f2f95ffb640 1 -- 172.21.15.3:0/3606315969 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644a0904a20 con 0x5644a09a97d0 2024-09-12T02:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.424+0000 7f2f95ffb640 10 client.26050 handle_command_reply: tid=0 2024-09-12T02:24:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.424+0000 7f2fb7841740 1 client.26050 shutdown 2024-09-12T02:24:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.432+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 msgr2=0x5644a09a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.432+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009ca0 tx=0x7f2fa802a8d0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.432+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 shutdown_connections 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a09a97d0 0x5644a09abbc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a0999370 0x5644a09a5150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09942e0 0x5644a099e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3606315969 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a099dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 >> 172.21.15.3:0/3606315969 conn(0x5644a098f010 msgr2=0x5644a0990820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.433+0000 7f2f95ffb640 0 client.26050 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.435+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 shutdown_connections 2024-09-12T02:24:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.436+0000 7f2fb7841740 1 -- 172.21.15.3:0/3606315969 wait complete. 2024-09-12T02:24:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.436+0000 7f2fb7841740 20 client.26050 trim_cache size 0 max 16384 2024-09-12T02:24:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 Processor -- start 2024-09-12T02:24:18.469 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 -- start start 2024-09-12T02:24:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09939a0 0x5644a098fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a0999370 0x5644a0990400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.470 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b7ef0 con 0x5644a0999370 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b9a30 con 0x5644a09939a0 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fb7841740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b85c0 con 0x5644a09942e0 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09939a0 0x5644a098fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56328/0 (socket says 172.21.15.3:56328) 2024-09-12T02:24:18.471 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.469+0000 7f2fadd9b640 1 -- 172.21.15.3:0/3588267311 learned_addr learned my addr 172.21.15.3:0/3588267311 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 -- 172.21.15.3:0/3588267311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09939a0 msgr2=0x5644a098fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09939a0 0x5644a098fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 -- 172.21.15.3:0/3588267311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a0999370 msgr2=0x5644a0990400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a0999370 0x5644a0990400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 -- 172.21.15.3:0/3588267311 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a0994f70 con 0x5644a09942e0 2024-09-12T02:24:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.470+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5644a0994d10 tx=0x7f2f98005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83f79708d7dd54dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.471+0000 7f2f97fff640 1 -- 172.21.15.3:0/3588267311 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980052c0 con 0x5644a09942e0 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.471+0000 7f2f97fff640 1 -- 172.21.15.3:0/3588267311 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9800bdd0 con 0x5644a09942e0 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.471+0000 7f2f97fff640 1 -- 172.21.15.3:0/3588267311 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980116a0 con 0x5644a09942e0 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 -- 172.21.15.3:0/3588267311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 msgr2=0x5644a09a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5644a0994d10 tx=0x7f2f98005880 comp rx=0 tx=0).stop 2024-09-12T02:24:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 -- 172.21.15.3:0/3588267311 shutdown_connections 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a0999370 0x5644a0990400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a5110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 --2- 172.21.15.3:0/3588267311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a09939a0 0x5644a098fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 -- 172.21.15.3:0/3588267311 >> 172.21.15.3:0/3588267311 conn(0x5644a098f010 msgr2=0x5644a0995480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 -- 172.21.15.3:0/3588267311 shutdown_connections 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.472+0000 7f2fb7841740 1 -- 172.21.15.3:0/3588267311 wait complete. 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.473+0000 7f2fb7841740 1 -- start start 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.473+0000 7f2fb7841740 1 Processor -- start 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a09a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fb7841740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a09a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56330/0 (socket says 172.21.15.3:56330) 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.474+0000 7f2fadd9b640 1 -- 172.21.15.3:0/2874196306 learned_addr learned my addr 172.21.15.3:0/2874196306 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b9360 con 0x5644a09939a0 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b8c90 con 0x5644a0999370 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb01b5ed0 con 0x5644a09942e0 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2faed9d640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a09a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fb7841740 10 client.0 _ll_register_callbacks cb 0x7f2fa80316f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fadd9b640 1 -- 172.21.15.3:0/2874196306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 msgr2=0x5644a09a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a09a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fadd9b640 1 -- 172.21.15.3:0/2874196306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 msgr2=0x5644a09a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a09a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.475+0000 7f2fadd9b640 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a09a3fe0 con 0x5644a09942e0 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.476+0000 7f2fadd9b640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2f98010fe0 tx=0x7f2f98003890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a73b7390c35511ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.476+0000 7f2f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.477+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98036060 con 0x5644a09942e0 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.477+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f2f98005050 con 0x5644a09942e0 2024-09-12T02:24:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.478+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f9800bdd0 con 0x5644a09942e0 2024-09-12T02:24:18.480 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.478+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f9802d040 con 0x5644a09942e0 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.479+0000 7f2fb7841740 10 client.26056 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.479+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5644a09a52b0 con 0x5644a09942e0 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9800bdd0 con 0x5644a09942e0 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f2f98030070 con 0x5644a09942e0 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2fb7841740 10 client.26056 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2fb7841740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2fb7841740 20 client.26056 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a0a14890 0x5644a0a16cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.480+0000 7f2fb7841740 4 client.26056 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.481+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644a08fed70 con 0x5644a0a14890 2024-09-12T02:24:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.481+0000 7f2fae59c640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a0a14890 0x5644a0a16cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.481+0000 7f2f95ffb640 10 client.26056 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.481+0000 7f2fae59c640 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a0a14890 0x5644a0a16cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f807383ce3e1492f server_cookie=13a7d8f31289650e in_seq=0 out_seq=0 2024-09-12T02:24:18.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:18 smithi026 ceph-mon[35496]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 128 MiB/s wr, 17.20k op/s 2024-09-12T02:24:18.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:18 smithi003 ceph-mon[31476]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 128 MiB/s wr, 17.20k op/s 2024-09-12T02:24:18.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:18 smithi092 ceph-mon[36089]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 128 MiB/s wr, 17.20k op/s 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2f95ffb640 1 -- 172.21.15.3:0/2874196306 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644a08fed70 con 0x5644a0a14890 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2f95ffb640 10 client.26056 handle_command_reply: tid=0 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 client.26056 shutdown 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 msgr2=0x5644a09a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2f98010fe0 tx=0x7f2f98003890 comp rx=0 tx=0).stop 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 shutdown_connections 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5644a0a14890 0x5644a0a16cb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5644a0999370 0x5644a09a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2f95ffb640 0 client.26056 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5644a09942e0 0x5644a09a2a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 --2- 172.21.15.3:0/2874196306 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5644a09939a0 0x5644a09a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 >> 172.21.15.3:0/2874196306 conn(0x5644a098f010 msgr2=0x5644a098d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 shutdown_connections 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 1 -- 172.21.15.3:0/2874196306 wait complete. 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.790+0000 7f2fb7841740 20 client.26056 trim_cache size 0 max 16384 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:24:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:24:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 msgr2=0x7f2f9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005e30 tx=0x7f2fa8031460 comp rx=0 tx=0).stop 2024-09-12T02:24:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 msgr2=0x7f2fb01a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb01a6b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c970 tx=0x7f2fa000ce30 comp rx=0 tx=0).stop 2024-09-12T02:24:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 shutdown_connections 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2f9c0a2b80 0x7f2f9c0a5040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fb0109930 0x7f2fb01ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2fb0073790 0x7f2fb01a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 --2- 172.21.15.3:0/1964134337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb0072d10 0x7f2fb01a6b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 >> 172.21.15.3:0/1964134337 conn(0x7f2fb006ecd0 msgr2=0x7f2fb01168e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 shutdown_connections 2024-09-12T02:24:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:18.791+0000 7f2fb49c4640 1 -- 172.21.15.3:0/1964134337 wait complete. 2024-09-12T02:24:20.514 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:20 smithi026 ceph-mon[35496]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 100 MiB/s wr, 13.44k op/s 2024-09-12T02:24:20.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:20 smithi003 ceph-mon[31476]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 100 MiB/s wr, 13.44k op/s 2024-09-12T02:24:20.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:20 smithi092 ceph-mon[36089]: pgmap v514: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 100 MiB/s wr, 13.44k op/s 2024-09-12T02:24:22.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:22 smithi026 ceph-mon[35496]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 94 MiB/s wr, 12.55k op/s 2024-09-12T02:24:22.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:22 smithi003 ceph-mon[31476]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 94 MiB/s wr, 12.55k op/s 2024-09-12T02:24:22.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:22 smithi092 ceph-mon[36089]: pgmap v515: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 94 MiB/s wr, 12.55k op/s 2024-09-12T02:24:24.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:24 smithi026 ceph-mon[35496]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 121 MiB/s wr, 16.05k op/s 2024-09-12T02:24:24.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:24 smithi003 ceph-mon[31476]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 121 MiB/s wr, 16.05k op/s 2024-09-12T02:24:24.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:24 smithi092 ceph-mon[36089]: pgmap v516: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 121 MiB/s wr, 16.05k op/s 2024-09-12T02:24:26.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:26 smithi026 ceph-mon[35496]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 94 MiB/s wr, 12.47k op/s 2024-09-12T02:24:26.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:26 smithi003 ceph-mon[31476]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 94 MiB/s wr, 12.47k op/s 2024-09-12T02:24:26.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:26 smithi092 ceph-mon[36089]: pgmap v517: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 94 MiB/s wr, 12.47k op/s 2024-09-12T02:24:28.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:28 smithi092 ceph-mon[36089]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 108 MiB/s wr, 14.17k op/s 2024-09-12T02:24:28.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:28 smithi026 ceph-mon[35496]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 108 MiB/s wr, 14.17k op/s 2024-09-12T02:24:28.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:28 smithi003 ceph-mon[31476]: pgmap v518: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 108 MiB/s wr, 14.17k op/s 2024-09-12T02:24:30.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:30 smithi092 ceph-mon[36089]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 95 MiB/s wr, 12.41k op/s 2024-09-12T02:24:30.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:30 smithi026 ceph-mon[35496]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 95 MiB/s wr, 12.41k op/s 2024-09-12T02:24:30.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:30 smithi003 ceph-mon[31476]: pgmap v519: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 95 MiB/s wr, 12.41k op/s 2024-09-12T02:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.982+0000 7efe26696640 1 -- 172.21.15.3:0/407179727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 msgr2=0x7efe20108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.982+0000 7efe26696640 1 --2- 172.21.15.3:0/407179727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe20108e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efe18009be0 tx=0x7efe1802a6f0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.988+0000 7efe26696640 1 -- 172.21.15.3:0/407179727 shutdown_connections 2024-09-12T02:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.988+0000 7efe26696640 1 --2- 172.21.15.3:0/407179727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe20108e80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.988+0000 7efe26696640 1 --2- 172.21.15.3:0/407179727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20101c30 0x7efe201020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.988+0000 7efe26696640 1 --2- 172.21.15.3:0/407179727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe20100a40 0x7efe20100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.988+0000 7efe26696640 1 -- 172.21.15.3:0/407179727 >> 172.21.15.3:0/407179727 conn(0x7efe200fc1c0 msgr2=0x7efe200fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.993+0000 7efe26696640 1 -- 172.21.15.3:0/407179727 shutdown_connections 2024-09-12T02:24:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.994+0000 7efe26696640 1 -- 172.21.15.3:0/407179727 wait complete. 2024-09-12T02:24:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.994+0000 7efe26696640 1 Processor -- start 2024-09-12T02:24:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.995+0000 7efe26696640 1 -- start start 2024-09-12T02:24:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.995+0000 7efe26696640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 0x7efe201115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.995+0000 7efe26696640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe20101c30 0x7efe20111b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe26696640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe2010fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe26696640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010a4e0 con 0x7efe201025f0 2024-09-12T02:24:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe26696640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010a360 con 0x7efe20100a40 2024-09-12T02:24:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe26696640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010a660 con 0x7efe20101c30 2024-09-12T02:24:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 0x7efe201115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 0x7efe201115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52506/0 (socket says 172.21.15.3:52506) 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe1ffff640 1 -- 172.21.15.3:0/71068961 learned_addr learned my addr 172.21.15.3:0/71068961 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.996+0000 7efe24c33640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe2010fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 -- 172.21.15.3:0/71068961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe20101c30 msgr2=0x7efe20111b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe20101c30 0x7efe20111b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 -- 172.21.15.3:0/71068961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 msgr2=0x7efe201115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 0x7efe201115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 -- 172.21.15.3:0/71068961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201103a0 con 0x7efe201025f0 2024-09-12T02:24:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:31.997+0000 7efe24c33640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe2010fc20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe18009b30 tx=0x7efe18031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.000+0000 7efe1d7fa640 1 -- 172.21.15.3:0/71068961 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18041020 con 0x7efe201025f0 2024-09-12T02:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.000+0000 7efe1d7fa640 1 -- 172.21.15.3:0/71068961 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe18031960 con 0x7efe201025f0 2024-09-12T02:24:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.000+0000 7efe1d7fa640 1 -- 172.21.15.3:0/71068961 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18036040 con 0x7efe201025f0 2024-09-12T02:24:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.002+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe20110630 con 0x7efe201025f0 2024-09-12T02:24:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.002+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201af150 con 0x7efe201025f0 2024-09-12T02:24:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.006+0000 7efe2942f740 1 Processor -- start 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 -- start start 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e7043da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e704b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555e6fa72f0 con 0x5555e7049370 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5555e6f4f270 con 0x5555e70439a0 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.014+0000 7efe2942f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555e6fa32b0 con 0x5555e70442e0 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37578/0 (socket says 172.21.15.3:37578) 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 -- 172.21.15.3:0/3810889123 learned_addr learned my addr 172.21.15.3:0/3810889123 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 -- 172.21.15.3:0/3810889123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 msgr2=0x5555e7043da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e7043da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 -- 172.21.15.3:0/3810889123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 msgr2=0x5555e704b830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e704b830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.015+0000 7efe1ffff640 1 -- 172.21.15.3:0/3810889123 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e6f4e340 con 0x5555e70442e0 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.016+0000 7efe1d7fa640 1 -- 172.21.15.3:0/71068961 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efe18044020 con 0x7efe201025f0 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.017+0000 7efe1ffff640 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d90 tx=0x7efe14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18de191c570d32e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.017+0000 7efe1d7fa640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 0x7efe080a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.017+0000 7efe1d7fa640 1 -- 172.21.15.3:0/71068961 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe180efb80 con 0x7efe201025f0 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.018+0000 7efe067fc640 1 -- 172.21.15.3:0/3810889123 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14004220 con 0x5555e70442e0 2024-09-12T02:24:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.018+0000 7efe067fc640 1 -- 172.21.15.3:0/3810889123 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe140043c0 con 0x5555e70442e0 2024-09-12T02:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.019+0000 7efe067fc640 1 -- 172.21.15.3:0/3810889123 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140199f0 con 0x5555e70442e0 2024-09-12T02:24:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.019+0000 7efe2942f740 1 -- 172.21.15.3:0/3810889123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 msgr2=0x5555e7048da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.019+0000 7efe2942f740 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d90 tx=0x7efe14011810 comp rx=0 tx=0).stop 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 -- 172.21.15.3:0/3810889123 shutdown_connections 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e704b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7048da0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 --2- 172.21.15.3:0/3810889123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e7043da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 -- 172.21.15.3:0/3810889123 >> 172.21.15.3:0/3810889123 conn(0x5555e703f010 msgr2=0x5555e703f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.021+0000 7efe2942f740 1 -- 172.21.15.3:0/3810889123 shutdown_connections 2024-09-12T02:24:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.022+0000 7efe2942f740 1 -- 172.21.15.3:0/3810889123 wait complete. 2024-09-12T02:24:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.022+0000 7efe2942f740 1 -- start start 2024-09-12T02:24:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 Processor -- start 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70439a0 0x5555e705ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70442e0 0x5555e705f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5555e705d3a0 con 0x5555e7049370 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x5555e6f30290 con 0x5555e70442e0 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5555e705d520 con 0x5555e70439a0 2024-09-12T02:24:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.023+0000 7efe2942f740 10 client.0 _ll_register_callbacks cb 0x5555e6f61140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.025+0000 7efe1f7fe640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 0x7efe080a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.025+0000 7efe24c33640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.025+0000 7efe24c33640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34036/0 (socket says 172.21.15.3:34036) 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.025+0000 7efe24c33640 1 -- 172.21.15.3:0/1614021872 learned_addr learned my addr 172.21.15.3:0/1614021872 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.027+0000 7efe1ffff640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70439a0 0x5555e705ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe1f7fe640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70442e0 0x5555e705f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe24c33640 1 -- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70439a0 msgr2=0x5555e705ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe24c33640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70439a0 0x5555e705ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe24c33640 1 -- 172.21.15.3:0/1614021872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70442e0 msgr2=0x5555e705f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe24c33640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70442e0 0x5555e705f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.028+0000 7efe24c33640 1 -- 172.21.15.3:0/1614021872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e6cba210 con 0x5555e7049370 2024-09-12T02:24:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.032+0000 7efe24c33640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efe180b87d0 tx=0x7efe180bbcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48c3a62726d150 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.032+0000 7efdff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:24:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.032+0000 7efe1f7fe640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 0x7efe080a5250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe10005d00 tx=0x7efe1000b470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.033+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180c0720 con 0x5555e7049370 2024-09-12T02:24:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.033+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe180c4b50 con 0x5555e7049370 2024-09-12T02:24:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.034+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe140043c0 con 0x5555e7049370 2024-09-12T02:24:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.035+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe180c4fe0 con 0x5555e7049370 2024-09-12T02:24:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.035+0000 7efe2942f740 10 client.16500 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.035+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5555e6d86620 con 0x5555e7049370 2024-09-12T02:24:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.037+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe140043c0 con 0x5555e7049370 2024-09-12T02:24:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.037+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7efe18004c50 con 0x5555e7049370 2024-09-12T02:24:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 10 client.16500 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 20 client.16500 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e706a900 0x5555e706ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 4 client.16500 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:24:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.038+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555e6fb4a20 con 0x5555e706a900 2024-09-12T02:24:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.042+0000 7efe1ffff640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e706a900 0x5555e706ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.049+0000 7efe1ffff640 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e706a900 0x5555e706ccf0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a80785ea6c7705 server_cookie=a3f5179c4ff05c76 in_seq=0 out_seq=0 2024-09-12T02:24:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.049+0000 7efdff7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.050+0000 7efdff7fe640 1 -- 172.21.15.3:0/1614021872 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555e6fb4a20 con 0x5555e706a900 2024-09-12T02:24:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.051+0000 7efdff7fe640 10 client.16500 handle_command_reply: tid=0 2024-09-12T02:24:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.051+0000 7efe2942f740 1 client.16500 shutdown 2024-09-12T02:24:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.054+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 msgr2=0x5555e7066150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.054+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efe180b87d0 tx=0x7efe180bbcd0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 shutdown_connections 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e706a900 0x5555e706ccf0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e7049370 0x5555e7066150 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70442e0 0x5555e705f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 --2- 172.21.15.3:0/1614021872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70439a0 0x5555e705ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.058+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 >> 172.21.15.3:0/1614021872 conn(0x5555e703f010 msgr2=0x5555e7040820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.059+0000 7efdff7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.059+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 shutdown_connections 2024-09-12T02:24:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.060+0000 7efe2942f740 1 -- 172.21.15.3:0/1614021872 wait complete. 2024-09-12T02:24:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.060+0000 7efe2942f740 20 client.16500 trim_cache size 0 max 16384 2024-09-12T02:24:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.102+0000 7efe2942f740 1 Processor -- start 2024-09-12T02:24:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.102+0000 7efe2942f740 1 -- start start 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe2942f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 0x5555e7048c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe2942f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7066110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe2942f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e7049370 0x5555e703fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe2942f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe201af710 con 0x5555e70439a0 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7066110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe1f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7066110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:37602/0 (socket says 172.21.15.3:37602) 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe1f7fe640 1 -- 172.21.15.3:0/2911074542 learned_addr learned my addr 172.21.15.3:0/2911074542 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe201b1250 con 0x5555e7049370 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.103+0000 7efe24c33640 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e7049370 0x5555e703fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe201afde0 con 0x5555e70442e0 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 0x5555e7048c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 -- 172.21.15.3:0/2911074542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 msgr2=0x5555e7066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7066110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 -- 172.21.15.3:0/2911074542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e7049370 msgr2=0x5555e703fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e7049370 0x5555e703fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.104+0000 7efe1ffff640 1 -- 172.21.15.3:0/2911074542 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555e70486d0 con 0x5555e70439a0 2024-09-12T02:24:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.105+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 0x5555e7048c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe14011d10 tx=0x7efe140045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cf8aa179f16a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.106+0000 7efe05ffb640 1 -- 172.21.15.3:0/2911074542 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14005cb0 con 0x5555e70439a0 2024-09-12T02:24:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.106+0000 7efe05ffb640 1 -- 172.21.15.3:0/2911074542 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe14009980 con 0x5555e70439a0 2024-09-12T02:24:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.107+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 msgr2=0x5555e7048c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.107+0000 7efe2942f740 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 0x5555e7048c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe14011d10 tx=0x7efe140045b0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.109+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 shutdown_connections 2024-09-12T02:24:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.109+0000 7efe2942f740 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e7049370 0x5555e703fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.109+0000 7efe2942f740 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e70442e0 0x5555e7066110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.110+0000 7efe2942f740 1 --2- 172.21.15.3:0/2911074542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70439a0 0x5555e7048c80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.110+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 >> 172.21.15.3:0/2911074542 conn(0x5555e703f010 msgr2=0x5555e70453c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.116+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 shutdown_connections 2024-09-12T02:24:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.116+0000 7efe2942f740 1 -- 172.21.15.3:0/2911074542 wait complete. 2024-09-12T02:24:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.116+0000 7efe2942f740 1 -- start start 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.116+0000 7efe2942f740 1 Processor -- start 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e70636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70442e0 0x5555e7063c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 0x5555e7064c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe201b0b80 con 0x5555e70442e0 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7efe201b04b0 con 0x5555e70439a0 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efe200727f0 con 0x5555e7049370 2024-09-12T02:24:32.118 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.117+0000 7efe2942f740 10 client.0 _ll_register_callbacks cb 0x7efe180bc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.118+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e70636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.118+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e70636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:52542/0 (socket says 172.21.15.3:52542) 2024-09-12T02:24:32.119 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.118+0000 7efe1ffff640 1 -- 172.21.15.3:0/2473866025 learned_addr learned my addr 172.21.15.3:0/2473866025 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.118+0000 7efe1f7fe640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70442e0 0x5555e7063c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.120 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.119+0000 7efe24c33640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 0x5555e7064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 -- 172.21.15.3:0/2473866025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 msgr2=0x5555e70636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e70636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 -- 172.21.15.3:0/2473866025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70442e0 msgr2=0x5555e7063c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70442e0 0x5555e7063c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 -- 172.21.15.3:0/2473866025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555e7065250 con 0x5555e7049370 2024-09-12T02:24:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.120+0000 7efe24c33640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 0x5555e7064c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe180bb990 tx=0x7efe1803cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664aeb75e826e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.123+0000 7efdff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:24:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.124+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe180c4d50 con 0x5555e7049370 2024-09-12T02:24:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.124+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe14009980 con 0x5555e7049370 2024-09-12T02:24:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.124+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7efe180b8160 con 0x5555e7049370 2024-09-12T02:24:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.126+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efe180c0720 con 0x5555e7049370 2024-09-12T02:24:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.130+0000 7efe2942f740 10 client.26071 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.130+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5555e70d55b0 con 0x5555e7049370 2024-09-12T02:24:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.132+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe14009980 con 0x5555e7049370 2024-09-12T02:24:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.133+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7efe180edad0 con 0x5555e7049370 2024-09-12T02:24:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 10 client.26071 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 20 client.26071 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e70d5bd0 0x5555e70d7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 4 client.26071 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:24:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.134+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555e6faed70 con 0x5555e70d5bd0 2024-09-12T02:24:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.138+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e70d5bd0 0x5555e70d7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.141+0000 7efe1ffff640 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e70d5bd0 0x5555e70d7ff0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a61179aaf58fb8 server_cookie=935342ab31312917 in_seq=0 out_seq=0 2024-09-12T02:24:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.141+0000 7efdff7fe640 10 client.26071 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efdff7fe640 1 -- 172.21.15.3:0/2473866025 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555e6faed70 con 0x5555e70d5bd0 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efdff7fe640 10 client.26071 handle_command_reply: tid=0 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 client.26071 shutdown 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 msgr2=0x5555e7064c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 0x5555e7064c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efe180bb990 tx=0x7efe1803cf10 comp rx=0 tx=0).stop 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 shutdown_connections 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5555e70d5bd0 0x5555e70d7ff0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5555e7049370 0x5555e7064c40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5555e70442e0 0x5555e7063c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 --2- 172.21.15.3:0/2473866025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5555e70439a0 0x5555e70636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 >> 172.21.15.3:0/2473866025 conn(0x5555e703f010 msgr2=0x5555e703d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efdff7fe640 0 client.26071 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 shutdown_connections 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.442+0000 7efe2942f740 1 -- 172.21.15.3:0/2473866025 wait complete. 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.443+0000 7efe2942f740 20 client.26071 trim_cache size 0 max 16384 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:24:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:24:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.445+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 msgr2=0x7efe080a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.445+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 0x7efe080a5250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efe10005d00 tx=0x7efe1000b470 comp rx=0 tx=0).stop 2024-09-12T02:24:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 msgr2=0x7efe2010fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe2010fc20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe18009b30 tx=0x7efe18031670 comp rx=0 tx=0).stop 2024-09-12T02:24:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 shutdown_connections 2024-09-12T02:24:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7efe080a2d90 0x7efe080a5250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe201025f0 0x7efe2010fc20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efe20101c30 0x7efe20111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 --2- 172.21.15.3:0/71068961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe20100a40 0x7efe201115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 >> 172.21.15.3:0/71068961 conn(0x7efe200fc1c0 msgr2=0x7efe20105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 shutdown_connections 2024-09-12T02:24:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:32.446+0000 7efe26696640 1 -- 172.21.15.3:0/71068961 wait complete. 2024-09-12T02:24:32.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:32 smithi026 ceph-mon[35496]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 96 MiB/s wr, 12.57k op/s 2024-09-12T02:24:32.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:32 smithi003 ceph-mon[31476]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 96 MiB/s wr, 12.57k op/s 2024-09-12T02:24:32.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:32 smithi092 ceph-mon[36089]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 96 MiB/s wr, 12.57k op/s 2024-09-12T02:24:34.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:34 smithi026 ceph-mon[35496]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 120 MiB/s wr, 15.67k op/s 2024-09-12T02:24:34.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:34 smithi003 ceph-mon[31476]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 120 MiB/s wr, 15.67k op/s 2024-09-12T02:24:34.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:34 smithi092 ceph-mon[36089]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 120 MiB/s wr, 15.67k op/s 2024-09-12T02:24:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:36 smithi026 ceph-mon[35496]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 91 MiB/s wr, 11.91k op/s 2024-09-12T02:24:36.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:36 smithi003 ceph-mon[31476]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 91 MiB/s wr, 11.91k op/s 2024-09-12T02:24:36.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:36 smithi092 ceph-mon[36089]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 91 MiB/s wr, 11.91k op/s 2024-09-12T02:24:38.671 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T02:24:38.671 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T02:24:38.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:24:38.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:38 smithi003 ceph-mon[31476]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 104 MiB/s wr, 13.59k op/s 2024-09-12T02:24:38.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:38 smithi026 ceph-mon[35496]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 104 MiB/s wr, 13.59k op/s 2024-09-12T02:24:38.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:38 smithi092 ceph-mon[36089]: pgmap v523: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 104 MiB/s wr, 13.59k op/s 2024-09-12T02:24:40.191 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:24:40.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:40 smithi026 ceph-mon[35496]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 108 MiB/s wr, 14.25k op/s 2024-09-12T02:24:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:40 smithi003 ceph-mon[31476]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 108 MiB/s wr, 14.25k op/s 2024-09-12T02:24:40.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:40 smithi092 ceph-mon[36089]: pgmap v524: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 108 MiB/s wr, 14.25k op/s 2024-09-12T02:24:42.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:42 smithi026 ceph-mon[35496]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 103 MiB/s wr, 13.71k op/s 2024-09-12T02:24:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:42 smithi003 ceph-mon[31476]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 103 MiB/s wr, 13.71k op/s 2024-09-12T02:24:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:42 smithi092 ceph-mon[36089]: pgmap v525: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 103 MiB/s wr, 13.71k op/s 2024-09-12T02:24:44.446 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:44 smithi003 ceph-mon[31476]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 116 MiB/s wr, 15.73k op/s 2024-09-12T02:24:44.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:44 smithi026 ceph-mon[35496]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 116 MiB/s wr, 15.73k op/s 2024-09-12T02:24:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:44 smithi092 ceph-mon[36089]: pgmap v526: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 116 MiB/s wr, 15.73k op/s 2024-09-12T02:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 -- 172.21.15.3:0/83041454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8100a40 msgr2=0x7f59e8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/83041454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8100a40 0x7f59e8100e40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009900 tx=0x7f59dc02a410 comp rx=0 tx=0).stop 2024-09-12T02:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 -- 172.21.15.3:0/83041454 shutdown_connections 2024-09-12T02:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/83041454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 0x7f59e8108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/83041454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8101c30 0x7f59e81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/83041454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8100a40 0x7f59e8100e40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 -- 172.21.15.3:0/83041454 >> 172.21.15.3:0/83041454 conn(0x7f59e80fc1c0 msgr2=0x7f59e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 -- 172.21.15.3:0/83041454 shutdown_connections 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.590+0000 7f59ef0e4640 1 -- 172.21.15.3:0/83041454 wait complete. 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 Processor -- start 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 -- start start 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8100a40 0x7f59e8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 0x7f59e819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810d810 con 0x7f59e8101c30 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810d690 con 0x7f59e81025f0 2024-09-12T02:24:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ef0e4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59e810d990 con 0x7f59e8100a40 2024-09-12T02:24:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46804/0 (socket says 172.21.15.3:46804) 2024-09-12T02:24:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 -- 172.21.15.3:0/1776371460 learned_addr learned my addr 172.21.15.3:0/1776371460 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ece80640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8100a40 0x7f59e8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59ed681640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 0x7f59e819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 -- 172.21.15.3:0/1776371460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8100a40 msgr2=0x7f59e8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8100a40 0x7f59e8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 -- 172.21.15.3:0/1776371460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 msgr2=0x7f59e819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 0x7f59e819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 -- 172.21.15.3:0/1776371460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e819fa90 con 0x7f59e8101c30 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.591+0000 7f59e7fff640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.592+0000 7f59e5ffb640 1 -- 172.21.15.3:0/1776371460 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d400cd80 con 0x7f59e8101c30 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.592+0000 7f59e5ffb640 1 -- 172.21.15.3:0/1776371460 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59d4005500 con 0x7f59e8101c30 2024-09-12T02:24:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.592+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e819fd80 con 0x7f59e8101c30 2024-09-12T02:24:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.592+0000 7f59e5ffb640 1 -- 172.21.15.3:0/1776371460 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4010690 con 0x7f59e8101c30 2024-09-12T02:24:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.592+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a0220 con 0x7f59e8101c30 2024-09-12T02:24:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.594+0000 7f59e5ffb640 1 -- 172.21.15.3:0/1776371460 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f59d40046c0 con 0x7f59e8101c30 2024-09-12T02:24:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 Processor -- start 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 -- start start 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdd6f0 0x556cecce21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556cecce2780 0x556cecce4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556cecc3c2f0 con 0x556cecce2780 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cecbe4270 con 0x556ceccdd6f0 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59f1e86740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556cecc382b0 con 0x556ceccdcdb0 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:41910/0 (socket says 172.21.15.3:41910) 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 -- 172.21.15.3:0/2728917508 learned_addr learned my addr 172.21.15.3:0/2728917508 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59e7fff640 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556cecce2780 0x556cecce4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59e5ffb640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ece80640 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdd6f0 0x556cecce21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 -- 172.21.15.3:0/2728917508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdd6f0 msgr2=0x556cecce21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59e5ffb640 1 -- 172.21.15.3:0/1776371460 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f59d40cf4d0 con 0x7f59e8101c30 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdd6f0 0x556cecce21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 -- 172.21.15.3:0/2728917508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556cecce2780 msgr2=0x556cecce4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556cecce2780 0x556cecce4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.595+0000 7f59ed681640 1 -- 172.21.15.3:0/2728917508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cecbe3340 con 0x556ceccdcdb0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.596+0000 7f59ed681640 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59e0013a50 tx=0x7f59e0013f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2ba6b2eeee261e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.596+0000 7f59ed681640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 0x7f59d80a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f59e000a000 tx=0x7f59e001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.596+0000 7f59d2ffd640 1 -- 172.21.15.3:0/2728917508 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0022070 con 0x556ceccdcdb0 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.596+0000 7f59d2ffd640 1 -- 172.21.15.3:0/2728917508 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e001cc50 con 0x556ceccdcdb0 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.596+0000 7f59d2ffd640 1 -- 172.21.15.3:0/2728917508 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0012740 con 0x556ceccdcdb0 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- 172.21.15.3:0/2728917508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 msgr2=0x556ceccdd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59e0013a50 tx=0x7f59e0013f10 comp rx=0 tx=0).stop 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- 172.21.15.3:0/2728917508 shutdown_connections 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556cecce2780 0x556cecce4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdd6f0 0x556cecce21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2728917508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdcdb0 0x556ceccdd1b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- 172.21.15.3:0/2728917508 >> 172.21.15.3:0/2728917508 conn(0x556ceccd4010 msgr2=0x556ceccd4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- 172.21.15.3:0/2728917508 shutdown_connections 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- 172.21.15.3:0/2728917508 wait complete. 2024-09-12T02:24:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 -- start start 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 Processor -- start 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.597+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556ceccee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59f1e86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x556cecce56f0 con 0x556ceccdd6f0 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59f1e86740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x556cecbc5290 con 0x556ceccdcdb0 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59f1e86740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556cecce5870 con 0x556cecce2780 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59f1e86740 10 client.0 _ll_register_callbacks cb 0x556cecbf6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ed681640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556ceccee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46826/0 (socket says 172.21.15.3:46826) 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59e7fff640 1 -- 172.21.15.3:0/735245219 learned_addr learned my addr 172.21.15.3:0/735245219 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:40788/0 (socket says 172.21.15.3:40788) 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 -- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 msgr2=0x556ceccee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556ceccee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 -- 172.21.15.3:0/735245219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 msgr2=0x556cecce7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 -- 172.21.15.3:0/735245219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cec94f210 con 0x556ceccdcdb0 2024-09-12T02:24:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59e7fff640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:24:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59ece80640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009230 tx=0x7f59dc0318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16fc9029e2ca559 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.598+0000 7f59c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:24:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc035040 con 0x556ceccdcdb0 2024-09-12T02:24:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e001cc50 con 0x556ceccdcdb0 2024-09-12T02:24:45.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f59dc004240 con 0x556ceccdcdb0 2024-09-12T02:24:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f59dc032e60 con 0x556ceccdcdb0 2024-09-12T02:24:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 10 client.25934 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556ceca1b620 con 0x556ceccdcdb0 2024-09-12T02:24:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59e001cc50 con 0x556ceccdcdb0 2024-09-12T02:24:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f59dc004b80 con 0x556ceccdcdb0 2024-09-12T02:24:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 10 client.25934 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 20 client.25934 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556ceccf5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 4 client.25934 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:24:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cecc49a20 con 0x556ceccf2c10 2024-09-12T02:24:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.599+0000 7f59e7fff640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556ceccf5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.600+0000 7f59c7fff640 10 client.25934 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.600+0000 7f59e7fff640 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556ceccf5000 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc53fa25694b544d server_cookie=1c701de817b6fd88 in_seq=0 out_seq=0 2024-09-12T02:24:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.600+0000 7f59c7fff640 1 -- 172.21.15.3:0/735245219 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cecc49a20 con 0x556ceccf2c10 2024-09-12T02:24:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.600+0000 7f59c7fff640 10 client.25934 handle_command_reply: tid=0 2024-09-12T02:24:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.600+0000 7f59f1e86740 1 client.25934 shutdown 2024-09-12T02:24:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 msgr2=0x556cecce6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009230 tx=0x7f59dc0318e0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 shutdown_connections 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556ceccf5000 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556ceccee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59c7fff640 0 client.25934 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecce7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 --2- 172.21.15.3:0/735245219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecce6f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 >> 172.21.15.3:0/735245219 conn(0x556ceccd4010 msgr2=0x556ceccd5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 shutdown_connections 2024-09-12T02:24:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 1 -- 172.21.15.3:0/735245219 wait complete. 2024-09-12T02:24:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.601+0000 7f59f1e86740 20 client.25934 trim_cache size 0 max 16384 2024-09-12T02:24:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 Processor -- start 2024-09-12T02:24:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 -- start start 2024-09-12T02:24:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdd6f0 0x556ceccee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cecce2780 0x556cecce7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a6d50 con 0x556ceccdcdb0 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a8890 con 0x556cecce2780 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59f1e86740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a7420 con 0x556ceccdd6f0 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46828/0 (socket says 172.21.15.3:46828) 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 -- 172.21.15.3:0/3853455117 learned_addr learned my addr 172.21.15.3:0/3853455117 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59e7fff640 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdd6f0 0x556ceccee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ed681640 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cecce2780 0x556cecce7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 -- 172.21.15.3:0/3853455117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdd6f0 msgr2=0x556ceccee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdd6f0 0x556ceccee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 -- 172.21.15.3:0/3853455117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cecce2780 msgr2=0x556cecce7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cecce2780 0x556cecce7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.616+0000 7f59ece80640 1 -- 172.21.15.3:0/3853455117 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ceccddbd0 con 0x556ceccdcdb0 2024-09-12T02:24:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59ece80640 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009850 tx=0x7f59dc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae41ffc0252a3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59d27fc640 1 -- 172.21.15.3:0/3853455117 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc035070 con 0x556ceccdcdb0 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59d27fc640 1 -- 172.21.15.3:0/3853455117 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59dc004c60 con 0x556ceccdcdb0 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59d27fc640 1 -- 172.21.15.3:0/3853455117 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc003b40 con 0x556ceccdcdb0 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 -- 172.21.15.3:0/3853455117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 msgr2=0x556cecce64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009850 tx=0x7f59dc031b10 comp rx=0 tx=0).stop 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 -- 172.21.15.3:0/3853455117 shutdown_connections 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556cecce2780 0x556cecce7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556ceccdd6f0 0x556ceccee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 --2- 172.21.15.3:0/3853455117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdcdb0 0x556cecce64b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 -- 172.21.15.3:0/3853455117 >> 172.21.15.3:0/3853455117 conn(0x556ceccd4010 msgr2=0x556cecce6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.617+0000 7f59f1e86740 1 -- 172.21.15.3:0/3853455117 shutdown_connections 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 -- 172.21.15.3:0/3853455117 wait complete. 2024-09-12T02:24:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 -- start start 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 Processor -- start 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecceb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556cecd5b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a81c0 con 0x556ceccdd6f0 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a7af0 con 0x556ceccdcdb0 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81a4d30 con 0x556cecce2780 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59f1e86740 10 client.0 _ll_register_callbacks cb 0x7f59dc032d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46840/0 (socket says 172.21.15.3:46840) 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 -- 172.21.15.3:0/2004087399 learned_addr learned my addr 172.21.15.3:0/2004087399 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59ece80640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecceb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59ed681640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556cecd5b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 -- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 msgr2=0x556cecd5b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556cecd5b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 -- 172.21.15.3:0/2004087399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 msgr2=0x556cecceb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecceb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59e7fff640 1 -- 172.21.15.3:0/2004087399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ceccee830 con 0x556ceccdd6f0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.618+0000 7f59ece80640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecceb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59e7fff640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f59d409b9c0 tx=0x7f59d409b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e0a7a8bebfbf2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40a0070 con 0x556ceccdd6f0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f59d40cf0f0 con 0x556ceccdd6f0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59dc004c60 con 0x556ceccdd6f0 2024-09-12T02:24:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f59d40a44a0 con 0x556ceccdd6f0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 10 client.16524 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556cecd5cf50 con 0x556ceccdd6f0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59dc004c60 con 0x556ceccdd6f0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f59d409b110 con 0x556ceccdd6f0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 10 client.16524 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 20 client.16524 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556cecd5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 4 client.16524 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.619+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cecc43d70 con 0x556ceccf2c10 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.620+0000 7f59ece80640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556cecd5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.620+0000 7f59ece80640 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556cecd5f8f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516d7a9a0f6398a3 server_cookie=62dcba138708787 in_seq=0 out_seq=0 2024-09-12T02:24:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.620+0000 7f59c7fff640 10 client.16524 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.817+0000 7f59c7fff640 1 -- 172.21.15.3:0/2004087399 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cecc43d70 con 0x556ceccf2c10 2024-09-12T02:24:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.817+0000 7f59c7fff640 10 client.16524 handle_command_reply: tid=0 2024-09-12T02:24:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.817+0000 7f59f1e86740 1 client.16524 shutdown 2024-09-12T02:24:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.817+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 msgr2=0x556cecceb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.817+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f59d409b9c0 tx=0x7f59d409b6e0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 shutdown_connections 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x556ceccf2c10 0x556cecd5f8f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556cecce2780 0x556cecd5b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59c7fff640 0 client.16524 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x556ceccdd6f0 0x556cecceb740 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 --2- 172.21.15.3:0/2004087399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556ceccdcdb0 0x556cecceb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 >> 172.21.15.3:0/2004087399 conn(0x556ceccd4010 msgr2=0x556ceccd29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 shutdown_connections 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 1 -- 172.21.15.3:0/2004087399 wait complete. 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.818+0000 7f59f1e86740 20 client.16524 trim_cache size 0 max 16384 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:24:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:24:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 msgr2=0x7f59d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 0x7f59d80a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f59e000a000 tx=0x7f59e001d000 comp rx=0 tx=0).stop 2024-09-12T02:24:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 msgr2=0x7f59e81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e9f0 tx=0x7f59d400edf0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 shutdown_connections 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e81025f0 0x7f59e819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59e8101c30 0x7f59e81983b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 --2- 172.21.15.3:0/1776371460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59e8100a40 0x7f59e8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 >> 172.21.15.3:0/1776371460 conn(0x7f59e80fc1c0 msgr2=0x7f59e8105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 shutdown_connections 2024-09-12T02:24:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:45.819+0000 7f59ef0e4640 1 -- 172.21.15.3:0/1776371460 wait complete. 2024-09-12T02:24:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:46 smithi026 ceph-mon[35496]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 97 MiB/s wr, 13.51k op/s 2024-09-12T02:24:46.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:46 smithi003 ceph-mon[31476]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 97 MiB/s wr, 13.51k op/s 2024-09-12T02:24:46.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:46 smithi092 ceph-mon[36089]: pgmap v527: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 97 MiB/s wr, 13.51k op/s 2024-09-12T02:24:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:48 smithi026 ceph-mon[35496]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 108 MiB/s wr, 15.16k op/s 2024-09-12T02:24:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:48 smithi003 ceph-mon[31476]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 108 MiB/s wr, 15.16k op/s 2024-09-12T02:24:48.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:48 smithi092 ceph-mon[36089]: pgmap v528: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 108 MiB/s wr, 15.16k op/s 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 -- 172.21.15.3:0/3992428912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102570 msgr2=0x7fe0641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 --2- 172.21.15.3:0/3992428912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102570 0x7fe0641029b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe05800d940 tx=0x7fe05802e650 comp rx=0 tx=0).stop 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 -- 172.21.15.3:0/3992428912 shutdown_connections 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 --2- 172.21.15.3:0/3992428912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064102ef0 0x7fe06410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 --2- 172.21.15.3:0/3992428912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102570 0x7fe0641029b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 --2- 172.21.15.3:0/3992428912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064100e20 0x7fe064101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 -- 172.21.15.3:0/3992428912 >> 172.21.15.3:0/3992428912 conn(0x7fe0640fc7d0 msgr2=0x7fe0640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:49.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 -- 172.21.15.3:0/3992428912 shutdown_connections 2024-09-12T02:24:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.768+0000 7fe069554640 1 -- 172.21.15.3:0/3992428912 wait complete. 2024-09-12T02:24:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 Processor -- start 2024-09-12T02:24:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 -- start start 2024-09-12T02:24:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:49.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064102570 0x7fe064110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:49.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102ef0 0x7fe064110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:49.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe064118790 con 0x7fe064100e20 2024-09-12T02:24:49.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe064118610 con 0x7fe064102ef0 2024-09-12T02:24:49.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.769+0000 7fe069554640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe064118910 con 0x7fe064102570 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.770+0000 7fe0627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064102570 0x7fe064110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56938/0 (socket says 172.21.15.3:56938) 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 -- 172.21.15.3:0/173661454 learned_addr learned my addr 172.21.15.3:0/173661454 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 -- 172.21.15.3:0/173661454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064102570 msgr2=0x7fe064110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064102570 0x7fe064110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 -- 172.21.15.3:0/173661454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102ef0 msgr2=0x7fe064110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102ef0 0x7fe064110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 -- 172.21.15.3:0/173661454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe064110b40 con 0x7fe064100e20 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.771+0000 7fe062ffd640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b4a0 tx=0x7fe05400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:49.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.772+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054014070 con 0x7fe064100e20 2024-09-12T02:24:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.772+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe064110e30 con 0x7fe064100e20 2024-09-12T02:24:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.772+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe064114e70 con 0x7fe064100e20 2024-09-12T02:24:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.772+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe054010af0 con 0x7fe064100e20 2024-09-12T02:24:49.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.773+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054017d10 con 0x7fe064100e20 2024-09-12T02:24:49.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.773+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050005290 con 0x7fe064100e20 2024-09-12T02:24:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.776+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe054017070 con 0x7fe064100e20 2024-09-12T02:24:49.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.776+0000 7fe04bfff640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.777+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe0540ce750 con 0x7fe064100e20 2024-09-12T02:24:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.777+0000 7fe0627fc640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:49.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.777+0000 7fe0627fc640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 0x7fe04c0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe058009230 tx=0x7fe05800f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:49.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:49.782+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe05409b6a0 con 0x7fe064100e20 2024-09-12T02:24:50.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.222+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe050005580 con 0x7fe064100e20 2024-09-12T02:24:50.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.223+0000 7fe04bfff640 1 -- 172.21.15.3:0/173661454 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11946 (secure 0 0 0) 0x7fe05409afa0 con 0x7fe064100e20 2024-09-12T02:24:50.224 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:24:50.224 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":37,"btime":"2024-09-12T02:22:15:807901+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:22:15.541633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:24:50.224 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 37 2024-09-12T02:24:50.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.227+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 msgr2=0x7fe04c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.227+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 0x7fe04c0a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe058009230 tx=0x7fe05800f040 comp rx=0 tx=0).stop 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.227+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 msgr2=0x7fe064116f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.227+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b4a0 tx=0x7fe05400b960 comp rx=0 tx=0).stop 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 shutdown_connections 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe04c0a2b80 0x7fe04c0a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe064102ef0 0x7fe064110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:50.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe064102570 0x7fe064110010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:50.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 --2- 172.21.15.3:0/173661454 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe064100e20 0x7fe064116f80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:50.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 >> 172.21.15.3:0/173661454 conn(0x7fe0640fc7d0 msgr2=0x7fe0641048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:50.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 shutdown_connections 2024-09-12T02:24:50.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:24:50.228+0000 7fe069554640 1 -- 172.21.15.3:0/173661454 wait complete. 2024-09-12T02:24:50.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:50 smithi003 ceph-mon[31476]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 105 MiB/s wr, 15.07k op/s 2024-09-12T02:24:50.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:50 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/173661454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:24:50.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:50 smithi026 ceph-mon[35496]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 105 MiB/s wr, 15.07k op/s 2024-09-12T02:24:50.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:50 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/173661454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:24:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:50 smithi092 ceph-mon[36089]: pgmap v529: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 105 MiB/s wr, 15.07k op/s 2024-09-12T02:24:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:50 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/173661454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:24:52.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:52 smithi026 ceph-mon[35496]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 78 MiB/s wr, 11.47k op/s 2024-09-12T02:24:52.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:52 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:24:52.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:52 smithi003 ceph-mon[31476]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 78 MiB/s wr, 11.47k op/s 2024-09-12T02:24:52.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:52 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:24:52.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:52 smithi092 ceph-mon[36089]: pgmap v530: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 78 MiB/s wr, 11.47k op/s 2024-09-12T02:24:52.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:52 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:24:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:54 smithi026 ceph-mon[35496]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 95 MiB/s wr, 13.95k op/s 2024-09-12T02:24:54.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:54 smithi003 ceph-mon[31476]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 95 MiB/s wr, 13.95k op/s 2024-09-12T02:24:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:54 smithi092 ceph-mon[36089]: pgmap v531: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 95 MiB/s wr, 13.95k op/s 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: Unable to set osd_memory_target on smithi003 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:24:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: Unable to set osd_memory_target on smithi003 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:24:55.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:24:55.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi003 to 177.0M 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: Unable to set osd_memory_target on smithi003 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:24:55.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:24:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:56 smithi026 ceph-mon[35496]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 83 MiB/s wr, 12.12k op/s 2024-09-12T02:24:56.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:56 smithi003 ceph-mon[31476]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 83 MiB/s wr, 12.12k op/s 2024-09-12T02:24:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:56 smithi092 ceph-mon[36089]: pgmap v532: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 83 MiB/s wr, 12.12k op/s 2024-09-12T02:24:58.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:24:58 smithi026 ceph-mon[35496]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 75 MiB/s wr, 10.88k op/s 2024-09-12T02:24:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:24:58 smithi003 ceph-mon[31476]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 75 MiB/s wr, 10.88k op/s 2024-09-12T02:24:58.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:24:58 smithi092 ceph-mon[36089]: pgmap v533: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 75 MiB/s wr, 10.88k op/s 2024-09-12T02:24:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 -- 172.21.15.3:0/3161275675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 msgr2=0x7f1a80100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/3161275675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80100fb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009900 tx=0x7f1a7402a410 comp rx=0 tx=0).stop 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 -- 172.21.15.3:0/3161275675 shutdown_connections 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/3161275675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a80108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/3161275675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80100fb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/3161275675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a80102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 -- 172.21.15.3:0/3161275675 >> 172.21.15.3:0/3161275675 conn(0x7f1a800fb790 msgr2=0x7f1a800fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 -- 172.21.15.3:0/3161275675 shutdown_connections 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 -- 172.21.15.3:0/3161275675 wait complete. 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.978+0000 7f1a7ffff640 1 Processor -- start 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 -- start start 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a801970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a8019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8010c520 con 0x7f1a80100b30 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8010c3a0 con 0x7f1a80101e90 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7ffff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8010c6a0 con 0x7f1a801014f0 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60348/0 (socket says 172.21.15.3:60348) 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 -- 172.21.15.3:0/810077309 learned_addr learned my addr 172.21.15.3:0/810077309 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a801970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a8019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 -- 172.21.15.3:0/810077309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 msgr2=0x7f1a801970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a801970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 -- 172.21.15.3:0/810077309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 msgr2=0x7f1a8019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a8019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 -- 172.21.15.3:0/810077309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8019e7d0 con 0x7f1a80100b30 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a801970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a8019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a7effd640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009860 tx=0x7f1a70009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.979+0000 7f1a67fff640 1 -- 172.21.15.3:0/810077309 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70004020 con 0x7f1a80100b30 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.980+0000 7f1a67fff640 1 -- 172.21.15.3:0/810077309 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a70007c80 con 0x7f1a80100b30 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.980+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a800ffa80 con 0x7f1a80100b30 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.980+0000 7f1a67fff640 1 -- 172.21.15.3:0/810077309 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70003b90 con 0x7f1a80100b30 2024-09-12T02:24:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.980+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a800ffff0 con 0x7f1a80100b30 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.982+0000 7f1a67fff640 1 -- 172.21.15.3:0/810077309 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1a70003d30 con 0x7f1a80100b30 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.982+0000 7f1a88275740 1 Processor -- start 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.982+0000 7f1a88275740 1 -- start start 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807eb70 0x55eb9807ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb98084540 0x55eb98086a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb97fe22f0 con 0x55eb9807f4b0 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb97f8a270 con 0x55eb98084540 2024-09-12T02:24:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a88275740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb97fde2b0 con 0x55eb9807eb70 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a67fff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 0x7f1a680a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807eb70 0x55eb9807ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60358/0 (socket says 172.21.15.3:60358) 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb98084540 0x55eb98086a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 -- 172.21.15.3:0/2185435940 learned_addr learned my addr 172.21.15.3:0/2185435940 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 0x7f1a680a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a67fff640 1 -- 172.21.15.3:0/810077309 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1a700d0510 con 0x7f1a80100b30 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 -- 172.21.15.3:0/2185435940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807eb70 msgr2=0x55eb9807ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807eb70 0x55eb9807ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 -- 172.21.15.3:0/2185435940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb98084540 msgr2=0x55eb98086a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb98084540 0x55eb98086a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 -- 172.21.15.3:0/2185435940 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb97f89340 con 0x55eb9807f4b0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7effd640 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1a70099730 tx=0x7f1a70009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e0798dfd1a2675 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 0x7f1a680a50b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009270 tx=0x7f1a78005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.983+0000 7f1a657fa640 1 -- 172.21.15.3:0/2185435940 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7009cd60 con 0x55eb9807f4b0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a657fa640 1 -- 172.21.15.3:0/2185435940 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a700a1110 con 0x55eb9807f4b0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a657fa640 1 -- 172.21.15.3:0/2185435940 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700d0330 con 0x55eb9807f4b0 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 -- 172.21.15.3:0/2185435940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 msgr2=0x55eb98083f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1a70099730 tx=0x7f1a70009f70 comp rx=0 tx=0).stop 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 -- 172.21.15.3:0/2185435940 shutdown_connections 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb98084540 0x55eb98086a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb98083f70 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 --2- 172.21.15.3:0/2185435940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807eb70 0x55eb9807ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 -- 172.21.15.3:0/2185435940 >> 172.21.15.3:0/2185435940 conn(0x55eb9807a1e0 msgr2=0x55eb9807a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 -- 172.21.15.3:0/2185435940 shutdown_connections 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.984+0000 7f1a88275740 1 -- 172.21.15.3:0/2185435940 wait complete. 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 -- start start 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 Processor -- start 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb98088dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98089300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55eb98087510 con 0x55eb98084540 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55eb97f6b290 con 0x55eb9807eb70 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55eb98087690 con 0x55eb9807f4b0 2024-09-12T02:24:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a7effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb98088dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60368/0 (socket says 172.21.15.3:60368) 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/528765184 learned_addr learned my addr 172.21.15.3:0/528765184 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.985+0000 7f1a88275740 10 client.0 _ll_register_callbacks cb 0x55eb97f9c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98089300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 msgr2=0x55eb98089300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98089300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/528765184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 msgr2=0x55eb98088dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb98088dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/528765184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb97cf5210 con 0x55eb98084540 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800ed60 tx=0x7f1a78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca3fcfcd1df5669 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7801f440 con 0x55eb98084540 2024-09-12T02:24:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1a7801f5e0 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a700a1110 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1a7801f7e0 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.986+0000 7f1a88275740 10 client.16539 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eb97dc1620 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a700a1110 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1a78032050 con 0x55eb98084540 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 10 client.16539 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 20 client.16539 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb98094ac0 0x55eb98096f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 4 client.16539 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.987+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb97fefa20 con 0x55eb98094ac0 2024-09-12T02:24:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.988+0000 7f1a7effd640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb98094ac0 0x55eb98096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.988+0000 7f1a7effd640 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb98094ac0 0x55eb98096f50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73bc0bc2ae94732 server_cookie=26bfd46edd289e7c in_seq=0 out_seq=0 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.988+0000 7f1a62ff5640 10 client.16539 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a62ff5640 1 -- 172.21.15.3:0/528765184 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb97fefa20 con 0x55eb98094ac0 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a62ff5640 10 client.16539 handle_command_reply: tid=0 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 client.16539 shutdown 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 msgr2=0x55eb98090340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800ed60 tx=0x7f1a78004a40 comp rx=0 tx=0).stop 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 shutdown_connections 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb98094ac0 0x55eb98096f50 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb98090340 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98089300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 --2- 172.21.15.3:0/528765184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb98088dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a62ff5640 0 client.16539 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 >> 172.21.15.3:0/528765184 conn(0x55eb9807a1e0 msgr2=0x55eb9807b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 shutdown_connections 2024-09-12T02:24:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 1 -- 172.21.15.3:0/528765184 wait complete. 2024-09-12T02:24:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:58.989+0000 7f1a88275740 20 client.16539 trim_cache size 0 max 16384 2024-09-12T02:24:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 Processor -- start 2024-09-12T02:24:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 -- start start 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9807b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98090260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a6d00 con 0x55eb98084540 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a8840 con 0x55eb9807eb70 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.004+0000 7f1a88275740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a73d0 con 0x55eb9807f4b0 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60382/0 (socket says 172.21.15.3:60382) 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/4102074918 learned_addr learned my addr 172.21.15.3:0/4102074918 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98090260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7effd640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9807b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/4102074918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 msgr2=0x55eb98090260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98090260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/4102074918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 msgr2=0x55eb9807b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9807b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 -- 172.21.15.3:0/4102074918 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9807a000 con 0x55eb98084540 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7effd640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9807b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009b50 tx=0x7f1a78000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563a0ddea21e804 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a64ff9640 1 -- 172.21.15.3:0/4102074918 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78038030 con 0x55eb98084540 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a64ff9640 1 -- 172.21.15.3:0/4102074918 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a78014950 con 0x55eb98084540 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.005+0000 7f1a64ff9640 1 -- 172.21.15.3:0/4102074918 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78033950 con 0x55eb98084540 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- 172.21.15.3:0/4102074918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 msgr2=0x55eb9807bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009b50 tx=0x7f1a78000f40 comp rx=0 tx=0).stop 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- 172.21.15.3:0/4102074918 shutdown_connections 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb98084540 0x55eb9807bb10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb9807f4b0 0x55eb98090260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 --2- 172.21.15.3:0/4102074918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9807b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- 172.21.15.3:0/4102074918 >> 172.21.15.3:0/4102074918 conn(0x55eb9807a1e0 msgr2=0x55eb980792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- 172.21.15.3:0/4102074918 shutdown_connections 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- 172.21.15.3:0/4102074918 wait complete. 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.006+0000 7f1a88275740 1 -- start start 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 Processor -- start 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9808ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb98084540 0x55eb9808f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a8170 con 0x55eb9807f4b0 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a7aa0 con 0x55eb9807eb70 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1a801a4ce0 con 0x55eb98084540 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a88275740 10 client.0 _ll_register_callbacks cb 0x7f1a7800ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60386/0 (socket says 172.21.15.3:60386) 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 -- 172.21.15.3:0/3132046259 learned_addr learned my addr 172.21.15.3:0/3132046259 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7f7fe640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb98084540 0x55eb9808f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7effd640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9808ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 -- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb98084540 msgr2=0x55eb9808f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb98084540 0x55eb9808f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 -- 172.21.15.3:0/3132046259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 msgr2=0x55eb9808ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:24:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9808ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 -- 172.21.15.3:0/3132046259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9808f910 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a7e7fc640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1a74031440 tx=0x7f1a74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614ea790607ccbae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.007+0000 7f1a62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74035040 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1a74005030 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a78014950 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1a74005230 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 10 client.16545 fetch_fsmap learned FSMap version 37 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eb980ffca0 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a78014950 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1a740035f0 con 0x55eb9807f4b0 2024-09-12T02:24:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 10 client.16545 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 20 client.16545 populate_metadata read hostname 'smithi003' 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb981005f0 0x55eb98102a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 4 client.16545 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.008+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb97fe9d70 con 0x55eb981005f0 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.009+0000 7f1a7effd640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb981005f0 0x55eb98102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.010+0000 7f1a62ff5640 10 client.16545 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.010+0000 7f1a7effd640 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb981005f0 0x55eb98102a10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a9c49e16b2065f server_cookie=451f66c30cac54d in_seq=0 out_seq=0 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.624+0000 7f1a62ff5640 1 -- 172.21.15.3:0/3132046259 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb97fe9d70 con 0x55eb981005f0 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a62ff5640 10 client.16545 handle_command_reply: tid=0 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 client.16545 shutdown 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 msgr2=0x55eb9808e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1a74031440 tx=0x7f1a74004270 comp rx=0 tx=0).stop 2024-09-12T02:24:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 shutdown_connections 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55eb981005f0 0x55eb98102a10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a62ff5640 0 client.16545 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55eb98084540 0x55eb9808f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55eb9807f4b0 0x55eb9808e2f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 --2- 172.21.15.3:0/3132046259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eb9807eb70 0x55eb9808ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 >> 172.21.15.3:0/3132046259 conn(0x55eb9807a1e0 msgr2=0x55eb9807a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 shutdown_connections 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.625+0000 7f1a88275740 1 -- 172.21.15.3:0/3132046259 wait complete. 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.626+0000 7f1a88275740 20 client.16545 trim_cache size 0 max 16384 2024-09-12T02:24:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:24:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:24:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:24:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:24:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.627+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 msgr2=0x7f1a680a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.627+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 0x7f1a680a50b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009270 tx=0x7f1a78005bd0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.627+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 msgr2=0x7f1a80196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.627+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009860 tx=0x7f1a70009d20 comp rx=0 tx=0).stop 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.628+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 shutdown_connections 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.628+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1a680a2bf0 0x7f1a680a50b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.628+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a80101e90 0x7f1a8019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.628+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1a801014f0 0x7f1a801970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.628+0000 7f1a7ffff640 1 --2- 172.21.15.3:0/810077309 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a80100b30 0x7f1a80196bb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.629+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 >> 172.21.15.3:0/810077309 conn(0x7f1a800fb790 msgr2=0x7f1a80104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.629+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 shutdown_connections 2024-09-12T02:24:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:24:59.629+0000 7f1a7ffff640 1 -- 172.21.15.3:0/810077309 wait complete. 2024-09-12T02:25:00.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:00 smithi026 ceph-mon[35496]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 82 MiB/s wr, 12.04k op/s 2024-09-12T02:25:00.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:00 smithi003 ceph-mon[31476]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 82 MiB/s wr, 12.04k op/s 2024-09-12T02:25:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:00 smithi092 ceph-mon[36089]: pgmap v534: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 82 MiB/s wr, 12.04k op/s 2024-09-12T02:25:02.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T02:25:02.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T02:25:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:02 smithi026 ceph-mon[35496]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 64 MiB/s wr, 9.63k op/s 2024-09-12T02:25:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:02 smithi003 ceph-mon[31476]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 64 MiB/s wr, 9.63k op/s 2024-09-12T02:25:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:02 smithi092 ceph-mon[36089]: pgmap v535: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 64 MiB/s wr, 9.63k op/s 2024-09-12T02:25:03.166 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:25:04.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:04 smithi003 ceph-mon[31476]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 79 MiB/s wr, 12.02k op/s 2024-09-12T02:25:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:04 smithi092 ceph-mon[36089]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 79 MiB/s wr, 12.02k op/s 2024-09-12T02:25:05.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:04 smithi026 ceph-mon[35496]: pgmap v536: 97 pgs: 97 active+clean; 13 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 79 MiB/s wr, 12.02k op/s 2024-09-12T02:25:06.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:06 smithi003 ceph-mon[31476]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-09-12T02:25:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:06 smithi092 ceph-mon[36089]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-09-12T02:25:07.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:06 smithi026 ceph-mon[35496]: pgmap v537: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 71 MiB/s wr, 10.93k op/s 2024-09-12T02:25:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:08 smithi003 ceph-mon[31476]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 67 MiB/s wr, 10.51k op/s 2024-09-12T02:25:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:08 smithi092 ceph-mon[36089]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 67 MiB/s wr, 10.51k op/s 2024-09-12T02:25:09.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:08 smithi026 ceph-mon[35496]: pgmap v538: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 67 MiB/s wr, 10.51k op/s 2024-09-12T02:25:10.729 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:10 smithi003 ceph-mon[31476]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 82 MiB/s wr, 12.87k op/s 2024-09-12T02:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.453+0000 7ff02f4fe640 1 -- 172.21.15.3:0/177419653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff02810e230 msgr2=0x7ff02810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.453+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/177419653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff02810e230 0x7ff02810e610 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff024009890 tx=0x7ff02402a370 comp rx=0 tx=0).stop 2024-09-12T02:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 -- 172.21.15.3:0/177419653 shutdown_connections 2024-09-12T02:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/177419653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff028108b50 0x7ff028115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/177419653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0281081f0 0x7ff028108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/177419653 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff02810e230 0x7ff02810e610 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 -- 172.21.15.3:0/177419653 >> 172.21.15.3:0/177419653 conn(0x7ff028077bb0 msgr2=0x7ff028079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 -- 172.21.15.3:0/177419653 shutdown_connections 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.454+0000 7ff02f4fe640 1 -- 172.21.15.3:0/177419653 wait complete. 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.455+0000 7ff02f4fe640 1 Processor -- start 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.455+0000 7ff02f4fe640 1 -- start start 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.455+0000 7ff02f4fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02f4fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff028108b50 0x7ff0281a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02f4fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff02810e230 0x7ff02819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02f4fe640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff028118580 con 0x7ff0281081f0 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02f4fe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff028118400 con 0x7ff02810e230 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02f4fe640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff028118700 con 0x7ff028108b50 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48406/0 (socket says 172.21.15.3:48406) 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 -- 172.21.15.3:0/3296739722 learned_addr learned my addr 172.21.15.3:0/3296739722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 -- 172.21.15.3:0/3296739722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff028108b50 msgr2=0x7ff0281a4bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff028108b50 0x7ff0281a4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 -- 172.21.15.3:0/3296739722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff02810e230 msgr2=0x7ff02819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:10.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02da9b640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff02810e230 0x7ff02819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff02810e230 0x7ff02819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 -- 172.21.15.3:0/3296739722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02819efd0 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.456+0000 7ff02d29a640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff024030910 tx=0x7ff024033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.457+0000 7ff01e7fc640 1 -- 172.21.15.3:0/3296739722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024037040 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.457+0000 7ff01e7fc640 1 -- 172.21.15.3:0/3296739722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024040070 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.457+0000 7ff01e7fc640 1 -- 172.21.15.3:0/3296739722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02403b690 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.457+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02819f260 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.457+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281ab510 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.459+0000 7ff01e7fc640 1 -- 172.21.15.3:0/3296739722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff024004350 con 0x7ff0281081f0 2024-09-12T02:25:10.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.460+0000 7ff01e7fc640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 0x7ff0140a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.460+0000 7ff01e7fc640 1 -- 172.21.15.3:0/3296739722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff0240ef110 con 0x7ff0281081f0 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.460+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 0x7ff0140a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.460+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 0x7ff0140a51d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff02810d920 tx=0x7ff01800b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 Processor -- start 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 -- start start 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e78e025160 con 0x55e78e0c1170 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e78dfcc720 con 0x55e78e0cba30 2024-09-12T02:25:10.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff0318d7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e78e021120 con 0x55e78e0c0850 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48412/0 (socket says 172.21.15.3:48412) 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 -- 172.21.15.3:0/1204395875 learned_addr learned my addr 172.21.15.3:0/1204395875 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02d29a640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02da9b640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 -- 172.21.15.3:0/1204395875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 msgr2=0x55e78e0c0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 -- 172.21.15.3:0/1204395875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 msgr2=0x55e78e0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.555+0000 7ff02ca99640 1 -- 172.21.15.3:0/1204395875 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e78dd96ba0 con 0x55e78e0c1170 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.556+0000 7ff02da9b640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.556+0000 7ff02ca99640 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff01800dda0 tx=0x7ff018010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433946c03dc3ff0d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.573+0000 7ff0137fe640 1 -- 172.21.15.3:0/1204395875 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01801d440 con 0x55e78e0c1170 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.573+0000 7ff0137fe640 1 -- 172.21.15.3:0/1204395875 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018022070 con 0x55e78e0c1170 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.573+0000 7ff0137fe640 1 -- 172.21.15.3:0/1204395875 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800f890 con 0x55e78e0c1170 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.574+0000 7ff0318d7740 1 -- 172.21.15.3:0/1204395875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 msgr2=0x55e78e0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.574+0000 7ff0318d7740 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff01800dda0 tx=0x7ff018010000 comp rx=0 tx=0).stop 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.576+0000 7ff0318d7740 1 -- 172.21.15.3:0/1204395875 shutdown_connections 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.576+0000 7ff0318d7740 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.576+0000 7ff0318d7740 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0cb4f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.576+0000 7ff0318d7740 1 --2- 172.21.15.3:0/1204395875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.576+0000 7ff0318d7740 1 -- 172.21.15.3:0/1204395875 >> 172.21.15.3:0/1204395875 conn(0x55e78e0bcde0 msgr2=0x55e78e0bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.577+0000 7ff0318d7740 1 -- 172.21.15.3:0/1204395875 shutdown_connections 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.577+0000 7ff0318d7740 1 -- 172.21.15.3:0/1204395875 wait complete. 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.578+0000 7ff0318d7740 1 -- start start 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.578+0000 7ff0318d7740 1 Processor -- start 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55e78e0ce600 con 0x55e78e0c1170 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55e78dfad900 con 0x55e78e0cba30 2024-09-12T02:25:10.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55e78e0ce780 con 0x55e78e0c0850 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff02ca99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff02ca99640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48416/0 (socket says 172.21.15.3:48416) 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff02ca99640 1 -- 172.21.15.3:0/2962015758 learned_addr learned my addr 172.21.15.3:0/2962015758 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff02da9b640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.579+0000 7ff0318d7740 10 client.0 _ll_register_callbacks cb 0x55e78dfab780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 -- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 msgr2=0x55e78e0c88b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c88b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 -- 172.21.15.3:0/2962015758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 msgr2=0x55e78e0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 -- 172.21.15.3:0/2962015758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e78dfedc50 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff02da9b640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0c9330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff020005fd0 tx=0x7ff02000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82282bc62e410a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.580+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018022070 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.585+0000 7ff010ff9640 1 -- 172.21.15.3:0/2962015758 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02001d060 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.585+0000 7ff010ff9640 1 -- 172.21.15.3:0/2962015758 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff02000d5f0 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.585+0000 7ff010ff9640 1 -- 172.21.15.3:0/2962015758 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff02000d7f0 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.586+0000 7ff0318d7740 10 client.25964 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.586+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55e78e0da190 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff010ff9640 1 -- 172.21.15.3:0/2962015758 <== mon.2 v2:172.21.15.26:3300/0 4 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff02000b040 con 0x55e78e0cba30 2024-09-12T02:25:10.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 10 client.25964 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 10 client.25964 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 20 client.25964 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e0db540 0x55e78e0dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 4 client.25964 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.588+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e78e032890 con 0x55e78e0db540 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.592+0000 7ff02d29a640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e0db540 0x55e78e0dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.597+0000 7ff010ff9640 10 client.25964 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.597+0000 7ff02d29a640 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e0db540 0x55e78e0dd930 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc39db40a41706f server_cookie=d5ddf3c6e5eab863 in_seq=0 out_seq=0 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.598+0000 7ff010ff9640 1 -- 172.21.15.3:0/2962015758 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e78e032890 con 0x55e78e0db540 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.598+0000 7ff010ff9640 10 client.25964 handle_command_reply: tid=0 2024-09-12T02:25:10.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.598+0000 7ff0318d7740 1 client.25964 shutdown 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 msgr2=0x55e78e0c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0c9330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff020005fd0 tx=0x7ff02000c740 comp rx=0 tx=0).stop 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 shutdown_connections 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e0db540 0x55e78e0dd930 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0cba30 0x55e78e0c9330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 --2- 172.21.15.3:0/2962015758 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c0850 0x55e78e0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff010ff9640 0 client.25964 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 >> 172.21.15.3:0/2962015758 conn(0x55e78e0bcde0 msgr2=0x55e78e0c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 shutdown_connections 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 1 -- 172.21.15.3:0/2962015758 wait complete. 2024-09-12T02:25:10.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.600+0000 7ff0318d7740 20 client.25964 trim_cache size 0 max 16384 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.636+0000 7ff0318d7740 1 Processor -- start 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.636+0000 7ff0318d7740 1 -- start start 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 0x55e78e0bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b3c80 con 0x55e78e0c1170 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b57c0 con 0x55e78e0c0850 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.637+0000 7ff0318d7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b4350 con 0x55e78e0cba30 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02da9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 0x55e78e0bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02da9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 0x55e78e0bc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:56790/0 (socket says 172.21.15.3:56790) 2024-09-12T02:25:10.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02da9b640 1 -- 172.21.15.3:0/3943372124 learned_addr learned my addr 172.21.15.3:0/3943372124 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02d29a640 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 -- 172.21.15.3:0/3943372124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 msgr2=0x55e78e0bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 0x55e78e0bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 -- 172.21.15.3:0/3943372124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 msgr2=0x55e78e0c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.638+0000 7ff02ca99640 1 -- 172.21.15.3:0/3943372124 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e78e0c7860 con 0x55e78e0c1170 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.639+0000 7ff02ca99640 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0bbdb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55e78e0c9ef0 tx=0x7ff01800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a449c9ef3fb6207 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.642+0000 7ff012ffd640 1 -- 172.21.15.3:0/3943372124 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018005040 con 0x55e78e0c1170 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.642+0000 7ff012ffd640 1 -- 172.21.15.3:0/3943372124 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018016030 con 0x55e78e0c1170 2024-09-12T02:25:10.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.642+0000 7ff012ffd640 1 -- 172.21.15.3:0/3943372124 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01801d440 con 0x55e78e0c1170 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.643+0000 7ff0318d7740 1 -- 172.21.15.3:0/3943372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 msgr2=0x55e78e0bbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.643+0000 7ff0318d7740 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0bbdb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55e78e0c9ef0 tx=0x7ff01800d7d0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.643+0000 7ff0318d7740 1 -- 172.21.15.3:0/3943372124 shutdown_connections 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.644+0000 7ff0318d7740 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0cba30 0x55e78e0bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.644+0000 7ff0318d7740 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0c1170 0x55e78e0bbdb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.644+0000 7ff0318d7740 1 --2- 172.21.15.3:0/3943372124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.644+0000 7ff0318d7740 1 -- 172.21.15.3:0/3943372124 >> 172.21.15.3:0/3943372124 conn(0x55e78e0bcde0 msgr2=0x55e78e0bb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.646+0000 7ff0318d7740 1 -- 172.21.15.3:0/3943372124 shutdown_connections 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.646+0000 7ff0318d7740 1 -- 172.21.15.3:0/3943372124 wait complete. 2024-09-12T02:25:10.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.646+0000 7ff0318d7740 1 -- start start 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 Processor -- start 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 0x55e78e0d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0cba30 0x55e78e0d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b50f0 con 0x55e78e0cba30 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b4a20 con 0x55e78e0c0850 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b1c60 con 0x55e78e0c1170 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.647+0000 7ff0318d7740 10 client.0 _ll_register_callbacks cb 0x7ff028069920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.648+0000 7ff02d29a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.649+0000 7ff02d29a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:34668/0 (socket says 172.21.15.3:34668) 2024-09-12T02:25:10.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.649+0000 7ff02d29a640 1 -- 172.21.15.3:0/4248311015 learned_addr learned my addr 172.21.15.3:0/4248311015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.649+0000 7ff02da9b640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0cba30 0x55e78e0d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.649+0000 7ff02ca99640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 0x55e78e0d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.650+0000 7ff02ca99640 1 -- 172.21.15.3:0/4248311015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 msgr2=0x55e78e0d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.650+0000 7ff02ca99640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.650+0000 7ff02ca99640 1 -- 172.21.15.3:0/4248311015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0cba30 msgr2=0x55e78e0d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.650+0000 7ff02ca99640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0cba30 0x55e78e0d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.650+0000 7ff02ca99640 1 -- 172.21.15.3:0/4248311015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e78e0c0fa0 con 0x55e78e0c1170 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.651+0000 7ff02ca99640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 0x55e78e0d0140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55e78e0bd460 tx=0x7ff01800dda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c436346e748c8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.651+0000 7ff010ff9640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.651+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018004680 con 0x55e78e0c1170 2024-09-12T02:25:10.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.651+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7ff018004820 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.656+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018016030 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.662+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff018004a20 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.663+0000 7ff0318d7740 10 client.26119 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.663+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e78e13ec20 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.664+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018016030 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.664+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7ff018043760 con 0x55e78e0c1170 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 10 client.26119 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 10 client.26119 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 20 client.26119 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e13f110 0x55e78e141500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 4 client.26119 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T02:25:10.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.668+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55e78e02cbe0 con 0x55e78e13f110 2024-09-12T02:25:10.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.669+0000 7ff02d29a640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e13f110 0x55e78e141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:10.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.691+0000 7ff010ff9640 10 client.26119 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:10.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:10.691+0000 7ff02d29a640 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e13f110 0x55e78e141500 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a88d64a11658cd server_cookie=115e12e528dcf8a9 in_seq=0 out_seq=0 2024-09-12T02:25:10.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:10 smithi092 ceph-mon[36089]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 82 MiB/s wr, 12.87k op/s 2024-09-12T02:25:11.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:10 smithi026 ceph-mon[35496]: pgmap v539: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 82 MiB/s wr, 12.87k op/s 2024-09-12T02:25:12.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:12 smithi092 ceph-mon[36089]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 67 MiB/s wr, 10.45k op/s 2024-09-12T02:25:12.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.794+0000 7f5069601640 1 -- 172.21.15.3:0/2145117202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064100a40 msgr2=0x7f5064100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.794+0000 7f5069601640 1 --2- 172.21.15.3:0/2145117202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064100a40 0x7f5064100e40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f505800d910 tx=0x7f505802c390 comp rx=0 tx=0).stop 2024-09-12T02:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.800+0000 7f5069601640 1 -- 172.21.15.3:0/2145117202 shutdown_connections 2024-09-12T02:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.800+0000 7f5069601640 1 --2- 172.21.15.3:0/2145117202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50641025f0 0x7f5064108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.800+0000 7f5069601640 1 --2- 172.21.15.3:0/2145117202 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5064101c30 0x7f50641020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.800+0000 7f5069601640 1 --2- 172.21.15.3:0/2145117202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064100a40 0x7f5064100e40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.800+0000 7f5069601640 1 -- 172.21.15.3:0/2145117202 >> 172.21.15.3:0/2145117202 conn(0x7f50640fc1c0 msgr2=0x7f50640fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.804+0000 7f5069601640 1 -- 172.21.15.3:0/2145117202 shutdown_connections 2024-09-12T02:25:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.804+0000 7f5069601640 1 -- 172.21.15.3:0/2145117202 wait complete. 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.805+0000 7f5069601640 1 Processor -- start 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 -- start start 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064101c30 0x7f506406c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50641025f0 0x7f50640735a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f506410ae10 con 0x7f50641025f0 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f506410ac90 con 0x7f5064100a40 2024-09-12T02:25:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.806+0000 7f5069601640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f506410af90 con 0x7f5064101c30 2024-09-12T02:25:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.807+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.807+0000 7f5062ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:34674/0 (socket says 172.21.15.3:34674) 2024-09-12T02:25:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.807+0000 7f5062ffd640 1 -- 172.21.15.3:0/2259383718 learned_addr learned my addr 172.21.15.3:0/2259383718 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.807+0000 7f50637fe640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50641025f0 0x7f50640735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f50627fc640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064101c30 0x7f506406c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f5062ffd640 1 -- 172.21.15.3:0/2259383718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064101c30 msgr2=0x7f506406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f5062ffd640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064101c30 0x7f506406c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f5062ffd640 1 -- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50641025f0 msgr2=0x7f50640735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f5062ffd640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50641025f0 0x7f50640735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.808+0000 7f5062ffd640 1 -- 172.21.15.3:0/2259383718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5064073c10 con 0x7f5064100a40 2024-09-12T02:25:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.812+0000 7f5062ffd640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f5058034810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.814+0000 7f504bfff640 1 -- 172.21.15.3:0/2259383718 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058034c10 con 0x7f5064100a40 2024-09-12T02:25:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.814+0000 7f504bfff640 1 -- 172.21.15.3:0/2259383718 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058041070 con 0x7f5064100a40 2024-09-12T02:25:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.814+0000 7f504bfff640 1 -- 172.21.15.3:0/2259383718 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058033620 con 0x7f5064100a40 2024-09-12T02:25:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.815+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5064073ea0 con 0x7f5064100a40 2024-09-12T02:25:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.817+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641af140 con 0x7f5064100a40 2024-09-12T02:25:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.822+0000 7f504bfff640 1 -- 172.21.15.3:0/2259383718 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f505800c050 con 0x7f5064100a40 2024-09-12T02:25:12.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.825+0000 7f506c474740 1 Processor -- start 2024-09-12T02:25:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.826+0000 7f506c474740 1 -- start start 2024-09-12T02:25:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.826+0000 7f506c474740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.826+0000 7f506c474740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.826+0000 7f506c474740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2248b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f506c474740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f22480c2f0 con 0x55f2248a89a0 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f506c474740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f2247b4270 con 0x55f2248ae370 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f506c474740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f2248082b0 con 0x55f2248a92e0 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48452/0 (socket says 172.21.15.3:48452) 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.827+0000 7f50637fe640 1 -- 172.21.15.3:0/139206530 learned_addr learned my addr 172.21.15.3:0/139206530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.829+0000 7f50637fe640 1 -- 172.21.15.3:0/139206530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 msgr2=0x55f2248adda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.829+0000 7f50637fe640 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248adda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.829+0000 7f50637fe640 1 -- 172.21.15.3:0/139206530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 msgr2=0x55f2248b0830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:25:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.829+0000 7f50637fe640 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2248b0830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.830+0000 7f50637fe640 1 -- 172.21.15.3:0/139206530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2247b3340 con 0x55f2248a89a0 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.830+0000 7f50637fe640 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f505c00b500 tx=0x7f505c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3b1c0e3e71f5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.830+0000 7f50497fa640 1 -- 172.21.15.3:0/139206530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c00a970 con 0x55f2248a89a0 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.831+0000 7f504bfff640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 0x7f504c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.831+0000 7f50497fa640 1 -- 172.21.15.3:0/139206530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505c019070 con 0x55f2248a89a0 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.831+0000 7f504bfff640 1 -- 172.21.15.3:0/2259383718 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f50580efb90 con 0x7f5064100a40 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.831+0000 7f50497fa640 1 -- 172.21.15.3:0/139206530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c013380 con 0x55f2248a89a0 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 -- 172.21.15.3:0/139206530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 msgr2=0x55f2248a8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f505c00b500 tx=0x7f505c009630 comp rx=0 tx=0).stop 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 -- 172.21.15.3:0/139206530 shutdown_connections 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2248b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 --2- 172.21.15.3:0/139206530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a8da0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.832+0000 7f506c474740 1 -- 172.21.15.3:0/139206530 >> 172.21.15.3:0/139206530 conn(0x55f2248a4010 msgr2=0x55f2248a4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.835+0000 7f506c474740 1 -- 172.21.15.3:0/139206530 shutdown_connections 2024-09-12T02:25:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.835+0000 7f5062ffd640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 0x7f504c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.836+0000 7f5062ffd640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 0x7f504c0a5210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f50580b8d50 tx=0x7f505802c390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.837+0000 7f506c474740 1 -- 172.21.15.3:0/139206530 wait complete. 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.837+0000 7f506c474740 1 -- start start 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.837+0000 7f506c474740 1 Processor -- start 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a89a0 0x55f2248b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f2248b13e0 con 0x55f2248ae370 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f224795290 con 0x55f2248a89a0 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f2248b1560 con 0x55f2248a92e0 2024-09-12T02:25:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f506c474740 10 client.0 _ll_register_callbacks cb 0x55f2247c6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48460/0 (socket says 172.21.15.3:48460) 2024-09-12T02:25:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.839+0000 7f50637fe640 1 -- 172.21.15.3:0/3094721324 learned_addr learned my addr 172.21.15.3:0/3094721324 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.840+0000 7f50637fe640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a89a0 0x55f2248b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.845+0000 7f50637fe640 1 -- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 msgr2=0x55f2248b3100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:25:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.845+0000 7f50637fe640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248b3100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.845+0000 7f50637fe640 1 -- 172.21.15.3:0/3094721324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a89a0 msgr2=0x55f2248b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.845+0000 7f50637fe640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a89a0 0x55f2248b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.845+0000 7f50637fe640 1 -- 172.21.15.3:0/3094721324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22451f210 con 0x55f2248ae370 2024-09-12T02:25:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.847+0000 7f50637fe640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55f2248b20f0 tx=0x7f505c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9fd3570f593ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.848+0000 7f5046ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.848+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c00abb0 con 0x55f2248ae370 2024-09-12T02:25:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.848+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f505c019070 con 0x55f2248ae370 2024-09-12T02:25:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.848+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f505c025030 con 0x55f2248ae370 2024-09-12T02:25:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.849+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f505c009740 con 0x55f2248ae370 2024-09-12T02:25:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.849+0000 7f506c474740 10 client.16575 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.849+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f2245eb620 con 0x55f2248ae370 2024-09-12T02:25:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.851+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505c019070 con 0x55f2248ae370 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.851+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f505c02c3f0 con 0x55f2248ae370 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.851+0000 7f506c474740 10 client.16575 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.851+0000 7f506c474740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.851+0000 7f506c474740 20 client.16575 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.852+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f2248cfca0 0x55f2248d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.852+0000 7f506c474740 4 client.16575 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.852+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f224819a20 con 0x55f2248cfca0 2024-09-12T02:25:12.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.858+0000 7f50627fc640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f2248cfca0 0x55f2248d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.878+0000 7f50627fc640 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f2248cfca0 0x55f2248d20e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9527a45667647e server_cookie=4d59b740c4b65015 in_seq=0 out_seq=0 2024-09-12T02:25:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.878+0000 7f5046ff5640 10 client.16575 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.880+0000 7f5046ff5640 1 -- 172.21.15.3:0/3094721324 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f224819a20 con 0x55f2248cfca0 2024-09-12T02:25:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.880+0000 7f5046ff5640 10 client.16575 handle_command_reply: tid=0 2024-09-12T02:25:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.880+0000 7f506c474740 1 client.16575 shutdown 2024-09-12T02:25:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.884+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 msgr2=0x55f2248ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.884+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55f2248b20f0 tx=0x7f505c009c40 comp rx=0 tx=0).stop 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 shutdown_connections 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f2248cfca0 0x55f2248d20e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248ae370 0x55f2248ba140 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a92e0 0x55f2248b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 --2- 172.21.15.3:0/3094721324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a89a0 0x55f2248b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 >> 172.21.15.3:0/3094721324 conn(0x55f2248a4010 msgr2=0x55f2248a5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.887+0000 7f5046ff5640 0 client.16575 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.893+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 shutdown_connections 2024-09-12T02:25:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.893+0000 7f506c474740 1 -- 172.21.15.3:0/3094721324 wait complete. 2024-09-12T02:25:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.894+0000 7f506c474740 20 client.16575 trim_cache size 0 max 16384 2024-09-12T02:25:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.929+0000 7f506c474740 1 Processor -- start 2024-09-12T02:25:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.929+0000 7f506c474740 1 -- start start 2024-09-12T02:25:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a92e0 0x55f2248ba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248ae370 0x55f2248a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50641af740 con 0x55f2248a89a0 2024-09-12T02:25:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50641b1280 con 0x55f2248a92e0 2024-09-12T02:25:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.930+0000 7f506c474740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f50641afe10 con 0x55f2248ae370 2024-09-12T02:25:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.933+0000 7f50637fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248ae370 0x55f2248a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.933+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.933+0000 7f50637fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48476/0 (socket says 172.21.15.3:48476) 2024-09-12T02:25:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.933+0000 7f50637fe640 1 -- 172.21.15.3:0/219985074 learned_addr learned my addr 172.21.15.3:0/219985074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.933+0000 7f50627fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a92e0 0x55f2248ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.936+0000 7f50637fe640 1 -- 172.21.15.3:0/219985074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248ae370 msgr2=0x55f2248a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.936+0000 7f50637fe640 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248ae370 0x55f2248a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.936+0000 7f50637fe640 1 -- 172.21.15.3:0/219985074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a92e0 msgr2=0x55f2248ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.936+0000 7f50637fe640 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a92e0 0x55f2248ba100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.936+0000 7f50637fe640 1 -- 172.21.15.3:0/219985074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2248d2760 con 0x55f2248a89a0 2024-09-12T02:25:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.940+0000 7f50637fe640 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f505c014970 tx=0x7f505c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ebf8be6480180c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.945+0000 7f5048ff9640 1 -- 172.21.15.3:0/219985074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c009740 con 0x55f2248a89a0 2024-09-12T02:25:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.945+0000 7f5048ff9640 1 -- 172.21.15.3:0/219985074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505c019070 con 0x55f2248a89a0 2024-09-12T02:25:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.945+0000 7f5048ff9640 1 -- 172.21.15.3:0/219985074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c02c570 con 0x55f2248a89a0 2024-09-12T02:25:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.949+0000 7f506c474740 1 -- 172.21.15.3:0/219985074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 msgr2=0x55f2248a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.949+0000 7f506c474740 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f505c014970 tx=0x7f505c000f40 comp rx=0 tx=0).stop 2024-09-12T02:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.950+0000 7f506c474740 1 -- 172.21.15.3:0/219985074 shutdown_connections 2024-09-12T02:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.950+0000 7f506c474740 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248ae370 0x55f2248a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.950+0000 7f506c474740 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248a92e0 0x55f2248ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.950+0000 7f506c474740 1 --2- 172.21.15.3:0/219985074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a89a0 0x55f2248a29a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.950+0000 7f506c474740 1 -- 172.21.15.3:0/219985074 >> 172.21.15.3:0/219985074 conn(0x55f2248a4010 msgr2=0x55f2248b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.951+0000 7f506c474740 1 -- 172.21.15.3:0/219985074 shutdown_connections 2024-09-12T02:25:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.952+0000 7f506c474740 1 -- 172.21.15.3:0/219985074 wait complete. 2024-09-12T02:25:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.952+0000 7f506c474740 1 -- start start 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.952+0000 7f506c474740 1 Processor -- start 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a89a0 0x55f2248b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2249389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50641b0bb0 con 0x55f2248a92e0 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f50641b04e0 con 0x55f2248ae370 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5064115360 con 0x55f2248a89a0 2024-09-12T02:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f506c474740 10 client.0 _ll_register_callbacks cb 0x7f505c009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f50627fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f50627fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48478/0 (socket says 172.21.15.3:48478) 2024-09-12T02:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f50627fc640 1 -- 172.21.15.3:0/2446829388 learned_addr learned my addr 172.21.15.3:0/2446829388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.953+0000 7f50637fe640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2249389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.954+0000 7f50627fc640 1 -- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a89a0 msgr2=0x55f2248b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.954+0000 7f50627fc640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a89a0 0x55f2248b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.954+0000 7f50627fc640 1 -- 172.21.15.3:0/2446829388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 msgr2=0x55f2249389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.954+0000 7f50627fc640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2249389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.954+0000 7f50627fc640 1 -- 172.21.15.3:0/2446829388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2248ba360 con 0x55f2248a92e0 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.955+0000 7f50627fc640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5054005d80 tx=0x7f505400d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ec77f521da1da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.955+0000 7f5046ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.955+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505401d020 con 0x55f2248a92e0 2024-09-12T02:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.955+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f505c019070 con 0x55f2248a92e0 2024-09-12T02:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.956+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f5054003d20 con 0x55f2248a92e0 2024-09-12T02:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.956+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f50540192c0 con 0x55f2248a92e0 2024-09-12T02:25:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.960+0000 7f506c474740 10 client.16581 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.960+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f22493a050 con 0x55f2248a92e0 2024-09-12T02:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f505c019070 con 0x55f2248a92e0 2024-09-12T02:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f5054011860 con 0x55f2248a92e0 2024-09-12T02:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 10 client.16581 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 20 client.16581 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f22493aa30 0x55f22493ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 4 client.16581 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.964+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f224813d70 con 0x55f22493aa30 2024-09-12T02:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.965+0000 7f50637fe640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f22493aa30 0x55f22493ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.968+0000 7f50637fe640 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f22493aa30 0x55f22493ce50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb60a5ae1ebafd56 server_cookie=328cd884af84634 in_seq=0 out_seq=0 2024-09-12T02:25:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:12.968+0000 7f5046ff5640 10 client.16581 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:13.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:12 smithi026 ceph-mon[35496]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 67 MiB/s wr, 10.45k op/s 2024-09-12T02:25:13.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:12 smithi003 ceph-mon[31476]: pgmap v540: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 67 MiB/s wr, 10.45k op/s 2024-09-12T02:25:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:14 smithi092 ceph-mon[36089]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 76 MiB/s wr, 11.91k op/s 2024-09-12T02:25:15.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:14 smithi026 ceph-mon[35496]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 76 MiB/s wr, 11.91k op/s 2024-09-12T02:25:15.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:14 smithi003 ceph-mon[31476]: pgmap v541: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 76 MiB/s wr, 11.91k op/s 2024-09-12T02:25:16.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f5046ff5640 1 -- 172.21.15.3:0/2446829388 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f224813d70 con 0x55f22493aa30 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f5046ff5640 10 client.16581 handle_command_reply: tid=0 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 client.16581 shutdown 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 msgr2=0x55f2248b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5054005d80 tx=0x7f505400d7a0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 shutdown_connections 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f22493aa30 0x55f22493ce50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f2248ae370 0x55f2249389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f2248a92e0 0x55f2248b72c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.257+0000 7f506c474740 1 --2- 172.21.15.3:0/2446829388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f2248a89a0 0x55f2248b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.258+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 >> 172.21.15.3:0/2446829388 conn(0x55f2248a4010 msgr2=0x55f2248a4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.258+0000 7f5046ff5640 0 client.16581 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.258+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 shutdown_connections 2024-09-12T02:25:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.258+0000 7f506c474740 1 -- 172.21.15.3:0/2446829388 wait complete. 2024-09-12T02:25:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.258+0000 7f506c474740 20 client.16581 trim_cache size 0 max 16384 2024-09-12T02:25:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:25:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:25:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:25:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 msgr2=0x7f504c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 0x7f504c0a5210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f50580b8d50 tx=0x7f505802c390 comp rx=0 tx=0).stop 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 msgr2=0x7f506406bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5058005ab0 tx=0x7f5058034810 comp rx=0 tx=0).stop 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 shutdown_connections 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f504c0a2d50 0x7f504c0a5210 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50641025f0 0x7f50640735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5064101c30 0x7f506406c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 --2- 172.21.15.3:0/2259383718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5064100a40 0x7f506406bff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 >> 172.21.15.3:0/2259383718 conn(0x7f50640fc1c0 msgr2=0x7f5064102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 shutdown_connections 2024-09-12T02:25:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:16.259+0000 7f5069601640 1 -- 172.21.15.3:0/2259383718 wait complete. 2024-09-12T02:25:17.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:16 smithi026 ceph-mon[35496]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 78 MiB/s wr, 12.52k op/s 2024-09-12T02:25:17.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:16 smithi003 ceph-mon[31476]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 78 MiB/s wr, 12.52k op/s 2024-09-12T02:25:17.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:16 smithi092 ceph-mon[36089]: pgmap v542: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 78 MiB/s wr, 12.52k op/s 2024-09-12T02:25:18.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:17 smithi026 ceph-mon[35496]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 68 MiB/s wr, 11.01k op/s 2024-09-12T02:25:18.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:17 smithi003 ceph-mon[31476]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 68 MiB/s wr, 11.01k op/s 2024-09-12T02:25:18.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:17 smithi092 ceph-mon[36089]: pgmap v543: 97 pgs: 97 active+clean; 13 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 68 MiB/s wr, 11.01k op/s 2024-09-12T02:25:20.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:19 smithi026 ceph-mon[35496]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 83 MiB/s wr, 13.82k op/s 2024-09-12T02:25:20.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:19 smithi003 ceph-mon[31476]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 83 MiB/s wr, 13.82k op/s 2024-09-12T02:25:20.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:19 smithi092 ceph-mon[36089]: pgmap v544: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 83 MiB/s wr, 13.82k op/s 2024-09-12T02:25:22.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:21 smithi026 ceph-mon[35496]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 77 MiB/s wr, 13.26k op/s 2024-09-12T02:25:22.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:21 smithi003 ceph-mon[31476]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 77 MiB/s wr, 13.26k op/s 2024-09-12T02:25:22.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:21 smithi092 ceph-mon[36089]: pgmap v545: 97 pgs: 97 active+clean; 13 GiB data, 43 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 77 MiB/s wr, 13.26k op/s 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff010ff9640 1 -- 172.21.15.3:0/4248311015 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e78e02cbe0 con 0x55e78e13f110 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff010ff9640 10 client.26119 handle_command_reply: tid=0 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 client.26119 shutdown 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 msgr2=0x55e78e0d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 0x55e78e0d0140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55e78e0bd460 tx=0x7ff01800dda0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 shutdown_connections 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55e78e13f110 0x55e78e141500 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55e78e0cba30 0x55e78e0d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55e78e0c1170 0x55e78e0d0140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 --2- 172.21.15.3:0/4248311015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e78e0c0850 0x55e78e0d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 >> 172.21.15.3:0/4248311015 conn(0x55e78e0bcde0 msgr2=0x55e78e0bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff010ff9640 0 client.26119 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 shutdown_connections 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.199+0000 7ff0318d7740 1 -- 172.21.15.3:0/4248311015 wait complete. 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff0318d7740 20 client.26119 trim_cache size 0 max 16384 2024-09-12T02:25:23.201 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 37, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 14, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "76d698d7": { 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 14, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:25:23.202 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 140.2, 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:23.203 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:25:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 msgr2=0x7ff0140a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 0x7ff0140a51d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff02810d920 tx=0x7ff01800b370 comp rx=0 tx=0).stop 2024-09-12T02:25:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 msgr2=0x7ff0281a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:23.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff024030910 tx=0x7ff024033d10 comp rx=0 tx=0).stop 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 shutdown_connections 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff0140a2d10 0x7ff0140a51d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff02810e230 0x7ff02819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff028108b50 0x7ff0281a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 --2- 172.21.15.3:0/3296739722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0281081f0 0x7ff0281a46b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 >> 172.21.15.3:0/3296739722 conn(0x7ff028077bb0 msgr2=0x7ff028079380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 shutdown_connections 2024-09-12T02:25:23.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:23.200+0000 7ff02f4fe640 1 -- 172.21.15.3:0/3296739722 wait complete. 2024-09-12T02:25:24.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:23 smithi026 ceph-mon[35496]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 89 MiB/s wr, 15.55k op/s 2024-09-12T02:25:24.027 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:23 smithi003 ceph-mon[31476]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 89 MiB/s wr, 15.55k op/s 2024-09-12T02:25:24.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:23 smithi092 ceph-mon[36089]: pgmap v546: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 89 MiB/s wr, 15.55k op/s 2024-09-12T02:25:26.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:25 smithi026 ceph-mon[35496]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 88 MiB/s wr, 15.58k op/s 2024-09-12T02:25:26.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:25 smithi003 ceph-mon[31476]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 88 MiB/s wr, 15.58k op/s 2024-09-12T02:25:26.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:25 smithi092 ceph-mon[36089]: pgmap v547: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 88 MiB/s wr, 15.58k op/s 2024-09-12T02:25:28.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:27 smithi026 ceph-mon[35496]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 72 MiB/s wr, 12.81k op/s 2024-09-12T02:25:28.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:27 smithi003 ceph-mon[31476]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 72 MiB/s wr, 12.81k op/s 2024-09-12T02:25:28.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:27 smithi092 ceph-mon[36089]: pgmap v548: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 72 MiB/s wr, 12.81k op/s 2024-09-12T02:25:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- 172.21.15.3:0/2022129050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 msgr2=0x7faaa0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/2022129050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0100e30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ca0 tx=0x7faa9002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- 172.21.15.3:0/2022129050 shutdown_connections 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/2022129050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa0108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/2022129050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 0x7faaa01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/2022129050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0100e30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- 172.21.15.3:0/2022129050 >> 172.21.15.3:0/2022129050 conn(0x7faaa00fc190 msgr2=0x7faaa00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- 172.21.15.3:0/2022129050 shutdown_connections 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- 172.21.15.3:0/2022129050 wait complete. 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 Processor -- start 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- start start 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 0x7faaa01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaa010c6b0 con 0x7faaa0100a30 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaa010c530 con 0x7faaa01025e0 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.364+0000 7faaa6ee4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faaa010c830 con 0x7faaa0101c20 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34780/0 (socket says 172.21.15.3:34780) 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 -- 172.21.15.3:0/3793985015 learned_addr learned my addr 172.21.15.3:0/3793985015 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa5481640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faa9ffff640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 0x7faaa01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 -- 172.21.15.3:0/3793985015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 msgr2=0x7faaa01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 0x7faaa01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 -- 172.21.15.3:0/3793985015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 msgr2=0x7faaa019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 -- 172.21.15.3:0/3793985015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa019fad0 con 0x7faaa0100a30 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa5481640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa4c80640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa90009bf0 tx=0x7faa90031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faa9dffb640 1 -- 172.21.15.3:0/3793985015 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9002fae0 con 0x7faaa0100a30 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faa9dffb640 1 -- 172.21.15.3:0/3793985015 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa9002fc80 con 0x7faaa0100a30 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa019fd60 con 0x7faaa0100a30 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.365+0000 7faa9dffb640 1 -- 172.21.15.3:0/3793985015 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa900467e0 con 0x7faaa0100a30 2024-09-12T02:25:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.366+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa01a0220 con 0x7faaa0100a30 2024-09-12T02:25:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.369+0000 7faa9dffb640 1 -- 172.21.15.3:0/3793985015 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faa90030b30 con 0x7faaa0100a30 2024-09-12T02:25:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.369+0000 7faaa9c83740 1 Processor -- start 2024-09-12T02:25:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.370+0000 7faa9dffb640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 0x7faa8c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.371+0000 7faa9dffb640 1 -- 172.21.15.3:0/3793985015 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7faa900f06d0 con 0x7faaa0100a30 2024-09-12T02:25:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.371+0000 7faaa9c83740 1 -- start start 2024-09-12T02:25:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b6f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b74780 0x55b4d2b76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2ace2f0 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2a76270 con 0x55b4d2b6edb0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa9c83740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2aca2b0 con 0x55b4d2b74780 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa5481640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 0x7faa8c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faa9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b6f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b6f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:44438/0 (socket says 172.21.15.3:44438) 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa4c80640 1 -- 172.21.15.3:0/4043945337 learned_addr learned my addr 172.21.15.3:0/4043945337 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.372+0000 7faaa5481640 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b74780 0x55b4d2b76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 -- 172.21.15.3:0/4043945337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b74780 msgr2=0x55b4d2b76c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b74780 0x55b4d2b76c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 -- 172.21.15.3:0/4043945337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 msgr2=0x55b4d2b6f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b6f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 -- 172.21.15.3:0/4043945337 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d2a75340 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faaa5481640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 0x7faa8c0a5140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa98009fd0 tx=0x7faa98011800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.373+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b741b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faa8800bbb0 tx=0x7faa8800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7cce5359ab2d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.374+0000 7faa96ffd640 1 -- 172.21.15.3:0/4043945337 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88015070 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.374+0000 7faa96ffd640 1 -- 172.21.15.3:0/4043945337 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa88005500 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.375+0000 7faaa9c83740 1 -- 172.21.15.3:0/4043945337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 msgr2=0x55b4d2b741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.375+0000 7faa96ffd640 1 -- 172.21.15.3:0/4043945337 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88015070 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.375+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b741b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7faa8800bbb0 tx=0x7faa8800f730 comp rx=0 tx=0).stop 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.375+0000 7faaa9c83740 1 -- 172.21.15.3:0/4043945337 shutdown_connections 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.376+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b74780 0x55b4d2b76c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.376+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b741b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.376+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4043945337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b6f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.376+0000 7faaa9c83740 1 -- 172.21.15.3:0/4043945337 >> 172.21.15.3:0/4043945337 conn(0x55b4d2b6a420 msgr2=0x55b4d2b6a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.376+0000 7faaa9c83740 1 -- 172.21.15.3:0/4043945337 shutdown_connections 2024-09-12T02:25:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.377+0000 7faaa9c83740 1 -- 172.21.15.3:0/4043945337 wait complete. 2024-09-12T02:25:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.377+0000 7faaa9c83740 1 -- start start 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.377+0000 7faaa9c83740 1 Processor -- start 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b777a0 con 0x55b4d2b74780 2024-09-12T02:25:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2a57290 con 0x55b4d2b6edb0 2024-09-12T02:25:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55b4d2b77920 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa9c83740 10 client.0 _ll_register_callbacks cb 0x55b4d2a88140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.378+0000 7faaa5481640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34800/0 (socket says 172.21.15.3:34800) 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 -- 172.21.15.3:0/77677188 learned_addr learned my addr 172.21.15.3:0/77677188 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa4c80640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faa9ffff640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 -- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 msgr2=0x55b4d2b795b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b795b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 -- 172.21.15.3:0/77677188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 msgr2=0x55b4d2b79070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b79070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.379+0000 7faaa5481640 1 -- 172.21.15.3:0/77677188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d27e1210 con 0x55b4d2b74780 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faaa5481640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faa98004500 tx=0x7faa98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29e08a4401e1595 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faa7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:29.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98025070 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7faa98021840 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa88005500 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.380+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faa9802f050 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faaa9c83740 10 client.16593 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b4d28ad620 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa88005500 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7faa9802a9b0 con 0x55b4d2b74780 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faaa9c83740 10 client.16593 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faaa9c83740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.381+0000 7faaa9c83740 20 client.16593 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.382+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2b960f0 0x55b4d2b98590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.382+0000 7faaa9c83740 4 client.16593 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.382+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d2adba20 con 0x55b4d2b960f0 2024-09-12T02:25:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.383+0000 7faaa4c80640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2b960f0 0x55b4d2b98590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.384+0000 7faaa4c80640 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2b960f0 0x55b4d2b98590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f054dfcae78d37 server_cookie=4c9e30a618a0572c in_seq=0 out_seq=0 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.384+0000 7faa7ffff640 10 client.16593 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.384+0000 7faa7ffff640 1 -- 172.21.15.3:0/77677188 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4d2adba20 con 0x55b4d2b960f0 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.384+0000 7faa7ffff640 10 client.16593 handle_command_reply: tid=0 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 client.16593 shutdown 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 msgr2=0x55b4d2b80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faa98004500 tx=0x7faa98004730 comp rx=0 tx=0).stop 2024-09-12T02:25:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 shutdown_connections 2024-09-12T02:25:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2b960f0 0x55b4d2b98590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b74780 0x55b4d2b80580 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b795b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 --2- 172.21.15.3:0/77677188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 >> 172.21.15.3:0/77677188 conn(0x55b4d2b6a420 msgr2=0x55b4d2b651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faa7ffff640 0 client.16593 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 shutdown_connections 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.385+0000 7faaa9c83740 1 -- 172.21.15.3:0/77677188 wait complete. 2024-09-12T02:25:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.386+0000 7faaa9c83740 20 client.16593 trim_cache size 0 max 16384 2024-09-12T02:25:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.414+0000 7faaa9c83740 1 Processor -- start 2024-09-12T02:25:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.414+0000 7faaa9c83740 1 -- start start 2024-09-12T02:25:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b77d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0072520 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0074060 con 0x55b4d2b74780 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faaa9c83740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0072bf0 con 0x55b4d2b6edb0 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faa9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faa9ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34804/0 (socket says 172.21.15.3:34804) 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.415+0000 7faa9ffff640 1 -- 172.21.15.3:0/4054272097 learned_addr learned my addr 172.21.15.3:0/4054272097 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faaa4c80640 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b77d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faaa5481640 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 -- 172.21.15.3:0/4054272097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6edb0 msgr2=0x55b4d2b77d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b77d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 -- 172.21.15.3:0/4054272097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 msgr2=0x55b4d2b6b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b6b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 -- 172.21.15.3:0/4054272097 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d2b6fbd0 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.416+0000 7faa9ffff640 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55b4d2b70120 tx=0x7faa8800b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a4868e606bde8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.417+0000 7faa967fc640 1 -- 172.21.15.3:0/4054272097 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88028070 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.417+0000 7faa967fc640 1 -- 172.21.15.3:0/4054272097 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8800b7e0 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faa967fc640 1 -- 172.21.15.3:0/4054272097 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88015040 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 -- 172.21.15.3:0/4054272097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 msgr2=0x55b4d2b80540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55b4d2b70120 tx=0x7faa8800b6f0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 -- 172.21.15.3:0/4054272097 shutdown_connections 2024-09-12T02:25:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b80540 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 --2- 172.21.15.3:0/4054272097 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b77d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 -- 172.21.15.3:0/4054272097 >> 172.21.15.3:0/4054272097 conn(0x55b4d2b6a420 msgr2=0x55b4d2b79070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 -- 172.21.15.3:0/4054272097 shutdown_connections 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.418+0000 7faaa9c83740 1 -- 172.21.15.3:0/4054272097 wait complete. 2024-09-12T02:25:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 -- start start 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 Processor -- start 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b7da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0073990 con 0x55b4d2b6edb0 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7faaa00732c0 con 0x55b4d2b74780 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0070500 con 0x55b4d2b6f6f0 2024-09-12T02:25:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.419+0000 7faaa9c83740 10 client.0 _ll_register_callbacks cb 0x7faa980119c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b7da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:44466/0 (socket says 172.21.15.3:44466) 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 -- 172.21.15.3:0/3835273030 learned_addr learned my addr 172.21.15.3:0/3835273030 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa4c80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b7da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34808/0 (socket says 172.21.15.3:34808) 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faa9ffff640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 -- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 msgr2=0x55b4d2b7df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b7df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 -- 172.21.15.3:0/3835273030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 msgr2=0x55b4d2b7da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b7da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.420+0000 7faaa5481640 1 -- 172.21.15.3:0/3835273030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d2b7f590 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.421+0000 7faaa5481640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa98004500 tx=0x7faa9802ab50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bea36776b934354 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.421+0000 7faa7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.421+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98025040 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.421+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7faa98021a50 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.421+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4d2ad5d70 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.422+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faa9801a070 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.422+0000 7faaa9c83740 10 client.25997 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.422+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b4d2b808a0 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.422+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b4d2ad5d70 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7faa9803c060 con 0x55b4d2b74780 2024-09-12T02:25:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 10 client.25997 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 20 client.25997 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2c015d0 0x55b4d2c039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 4 client.25997 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:25:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4d2a52900 con 0x55b4d2c015d0 2024-09-12T02:25:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.423+0000 7faaa4c80640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2c015d0 0x55b4d2c039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.424+0000 7faa7ffff640 10 client.25997 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.424+0000 7faaa4c80640 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2c015d0 0x55b4d2c039f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e6af8b2884940d server_cookie=fcc4b47eb93cff20 in_seq=0 out_seq=0 2024-09-12T02:25:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.449+0000 7faa7ffff640 1 -- 172.21.15.3:0/3835273030 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4d2a52900 con 0x55b4d2c015d0 2024-09-12T02:25:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.449+0000 7faa7ffff640 10 client.25997 handle_command_reply: tid=0 2024-09-12T02:25:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.449+0000 7faaa9c83740 1 client.25997 shutdown 2024-09-12T02:25:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.449+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 msgr2=0x55b4d2b7ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faa98004500 tx=0x7faa9802ab50 comp rx=0 tx=0).stop 2024-09-12T02:25:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 shutdown_connections 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55b4d2c015d0 0x55b4d2c039f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4d2b74780 0x55b4d2b7ef80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b4d2b6f6f0 0x55b4d2b7df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 --2- 172.21.15.3:0/3835273030 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55b4d2b6edb0 0x55b4d2b7da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 >> 172.21.15.3:0/3835273030 conn(0x55b4d2b6a420 msgr2=0x55b4d2b649a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faa7ffff640 0 client.25997 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 shutdown_connections 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 1 -- 172.21.15.3:0/3835273030 wait complete. 2024-09-12T02:25:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.450+0000 7faaa9c83740 20 client.25997 trim_cache size 0 max 16384 2024-09-12T02:25:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:25:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:25:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:25:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 msgr2=0x7faa8c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 0x7faa8c0a5140 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa98009fd0 tx=0x7faa98011800 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 msgr2=0x7faaa0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa90009bf0 tx=0x7faa90031ee0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 shutdown_connections 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7faa8c0a2c80 0x7faa8c0a5140 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa01025e0 0x7faaa019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faaa0101c20 0x7faaa01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 --2- 172.21.15.3:0/3793985015 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faaa0100a30 0x7faaa0197eb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 >> 172.21.15.3:0/3793985015 conn(0x7faaa00fc190 msgr2=0x7faaa006a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 shutdown_connections 2024-09-12T02:25:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:29.451+0000 7faaa6ee4640 1 -- 172.21.15.3:0/3793985015 wait complete. 2024-09-12T02:25:30.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:29 smithi026 ceph-mon[35496]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 MiB/s wr, 13.50k op/s 2024-09-12T02:25:30.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:29 smithi003 ceph-mon[31476]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 MiB/s wr, 13.50k op/s 2024-09-12T02:25:30.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:29 smithi092 ceph-mon[36089]: pgmap v549: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 77 MiB/s wr, 13.50k op/s 2024-09-12T02:25:31.963 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:31 smithi003 ceph-mon[31476]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 61 MiB/s wr, 10.48k op/s 2024-09-12T02:25:31.998 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '76d698d7', quiesce took 53.3 seconds. Will release after: -33.15 2024-09-12T02:25:31.998 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:25:32.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:31 smithi026 ceph-mon[35496]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 61 MiB/s wr, 10.48k op/s 2024-09-12T02:25:32.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:31 smithi092 ceph-mon[36089]: pgmap v550: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 61 MiB/s wr, 10.48k op/s 2024-09-12T02:25:32.502 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:25:33.940 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:33 smithi003 ceph-mon[31476]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 49 MiB/s wr, 8.36k op/s 2024-09-12T02:25:34.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:33 smithi026 ceph-mon[35496]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 49 MiB/s wr, 8.36k op/s 2024-09-12T02:25:34.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:33 smithi092 ceph-mon[36089]: pgmap v551: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 49 MiB/s wr, 8.36k op/s 2024-09-12T02:25:36.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:35 smithi026 ceph-mon[35496]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 MiB/s wr, 7.84k op/s 2024-09-12T02:25:36.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:35 smithi003 ceph-mon[31476]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 MiB/s wr, 7.84k op/s 2024-09-12T02:25:36.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:35 smithi092 ceph-mon[36089]: pgmap v552: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 MiB/s wr, 7.84k op/s 2024-09-12T02:25:37.938 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:25:37.938 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":37,"btime":"2024-09-12T02:22:15:807901+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:22:15.541633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 -- 172.21.15.3:0/3602422027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241036e0 msgr2=0x7fee24107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 --2- 172.21.15.3:0/3602422027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241036e0 0x7fee24107b90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fee10009930 tx=0x7fee1002a420 comp rx=0 tx=0).stop 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 -- 172.21.15.3:0/3602422027 shutdown_connections 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 --2- 172.21.15.3:0/3602422027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee241082a0 0x7fee2410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 --2- 172.21.15.3:0/3602422027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241036e0 0x7fee24107b90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 --2- 172.21.15.3:0/3602422027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee24102d30 0x7fee24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 -- 172.21.15.3:0/3602422027 >> 172.21.15.3:0/3602422027 conn(0x7fee240fc7d0 msgr2=0x7fee240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 -- 172.21.15.3:0/3602422027 shutdown_connections 2024-09-12T02:25:37.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.745+0000 7fee2af44640 1 -- 172.21.15.3:0/3602422027 wait complete. 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 Processor -- start 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 -- start start 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24102d30 0x7fee24198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee241036e0 0x7fee24198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee24112060 con 0x7fee241082a0 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee24111ee0 con 0x7fee24102d30 2024-09-12T02:25:37.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee2af44640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee241121e0 con 0x7fee241036e0 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee294e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee294e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34822/0 (socket says 172.21.15.3:34822) 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee294e1640 1 -- 172.21.15.3:0/2444460847 learned_addr learned my addr 172.21.15.3:0/2444460847 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee23fff640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee241036e0 0x7fee24198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.746+0000 7fee28ce0640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24102d30 0x7fee24198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:37.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 -- 172.21.15.3:0/2444460847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee241036e0 msgr2=0x7fee24198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee241036e0 0x7fee24198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 -- 172.21.15.3:0/2444460847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24102d30 msgr2=0x7fee24198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24102d30 0x7fee24198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 -- 172.21.15.3:0/2444460847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2419d5b0 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee294e1640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00cad0 tx=0x7fee1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00ae20 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee1c005020 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2419d8a0 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00f6f0 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.747+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee241a5180 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.748+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24103110 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.749+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fee1c0051d0 con 0x7fee241082a0 2024-09-12T02:25:37.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.750+0000 7fee21ffb640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 0x7fee140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.750+0000 7fee28ce0640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 0x7fee140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.750+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fee1c0cefe0 con 0x7fee241082a0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.750+0000 7fee28ce0640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 0x7fee140a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee18005fd0 tx=0x7fee18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.754+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee1c014030 con 0x7fee241082a0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.968+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee2410a2b0 con 0x7fee241082a0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.968+0000 7fee21ffb640 1 -- 172.21.15.3:0/2444460847 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11946 (secure 0 0 0) 0x7fee1c09bfd0 con 0x7fee241082a0 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 37 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 msgr2=0x7fee140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 0x7fee140a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fee18005fd0 tx=0x7fee18005850 comp rx=0 tx=0).stop 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 msgr2=0x7fee2419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00cad0 tx=0x7fee1c00cf90 comp rx=0 tx=0).stop 2024-09-12T02:25:37.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 shutdown_connections 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fee140a2b80 0x7fee140a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee241082a0 0x7fee2419ced0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee241036e0 0x7fee24198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 --2- 172.21.15.3:0/2444460847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee24102d30 0x7fee24198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 >> 172.21.15.3:0/2444460847 conn(0x7fee240fc7d0 msgr2=0x7fee24105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 shutdown_connections 2024-09-12T02:25:37.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:36.971+0000 7fee2af44640 1 -- 172.21.15.3:0/2444460847 wait complete. 2024-09-12T02:25:37.944 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:37 smithi003 ceph-mon[31476]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 MiB/s wr, 6.08k op/s 2024-09-12T02:25:37.945 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:37 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2444460847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:25:38.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:37 smithi026 ceph-mon[35496]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 MiB/s wr, 6.08k op/s 2024-09-12T02:25:38.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:37 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2444460847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:25:38.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:37 smithi092 ceph-mon[36089]: pgmap v553: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 36 MiB/s wr, 6.08k op/s 2024-09-12T02:25:38.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:37 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2444460847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:25:40.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:39 smithi026 ceph-mon[35496]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 7.22k op/s 2024-09-12T02:25:40.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:39 smithi003 ceph-mon[31476]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 7.22k op/s 2024-09-12T02:25:40.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:39 smithi092 ceph-mon[36089]: pgmap v554: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 7.22k op/s 2024-09-12T02:25:42.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:41 smithi026 ceph-mon[35496]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 7.02k op/s 2024-09-12T02:25:42.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:41 smithi003 ceph-mon[31476]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 7.02k op/s 2024-09-12T02:25:42.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:41 smithi092 ceph-mon[36089]: pgmap v555: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 7.02k op/s 2024-09-12T02:25:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 -- 172.21.15.3:0/2711669727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e081025f0 msgr2=0x7f8e081049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/2711669727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e081025f0 0x7f8e081049e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009900 tx=0x7f8e0402a410 comp rx=0 tx=0).stop 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 -- 172.21.15.3:0/2711669727 shutdown_connections 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/2711669727 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e08107850 0x7f8e08109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/2711669727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08104f20 0x7f8e08107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/2711669727 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e081025f0 0x7f8e081049e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 -- 172.21.15.3:0/2711669727 >> 172.21.15.3:0/2711669727 conn(0x7f8e080fc1f0 msgr2=0x7f8e080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 -- 172.21.15.3:0/2711669727 shutdown_connections 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.551+0000 7f8e0fe67640 1 -- 172.21.15.3:0/2711669727 wait complete. 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 Processor -- start 2024-09-12T02:25:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 -- start start 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e081025f0 0x7f8e08197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08107850 0x7f8e0819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810d880 con 0x7f8e08104f20 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810d700 con 0x7f8e08107850 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0fe67640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810da00 con 0x7f8e081025f0 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39102/0 (socket says 172.21.15.3:39102) 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0d402640 1 -- 172.21.15.3:0/1003122983 learned_addr learned my addr 172.21.15.3:0/1003122983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e081025f0 0x7f8e08197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.552+0000 7f8e0e404640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08107850 0x7f8e0819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 -- 172.21.15.3:0/1003122983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e081025f0 msgr2=0x7f8e08197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e081025f0 0x7f8e08197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 -- 172.21.15.3:0/1003122983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08107850 msgr2=0x7f8e0819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08107850 0x7f8e0819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 -- 172.21.15.3:0/1003122983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0819fb80 con 0x7f8e08104f20 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9c0 tx=0x7f8df800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8dfeffd640 1 -- 172.21.15.3:0/1003122983 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df800cd80 con 0x7f8e08104f20 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8dfeffd640 1 -- 172.21.15.3:0/1003122983 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df8005500 con 0x7f8e08104f20 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0819fe70 con 0x7f8e08104f20 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8dfeffd640 1 -- 172.21.15.3:0/1003122983 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8010690 con 0x7f8e08104f20 2024-09-12T02:25:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.553+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081a0330 con 0x7f8e08104f20 2024-09-12T02:25:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.555+0000 7f8dfeffd640 1 -- 172.21.15.3:0/1003122983 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8df80046c0 con 0x7f8e08104f20 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 Processor -- start 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 -- start start 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082419a0 0x560408241da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408249830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5604081a52f0 con 0x5604082422e0 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56040814d270 con 0x560408247370 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e12bff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5604081a12b0 con 0x5604082419a0 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39118/0 (socket says 172.21.15.3:39118) 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.556+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3850635446 learned_addr learned my addr 172.21.15.3:0/3850635446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8dfeffd640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 0x7f8df40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0d402640 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408249830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0e404640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 0x7f8df40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3850635446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082419a0 msgr2=0x560408241da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082419a0 0x560408241da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3850635446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 msgr2=0x560408249830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408249830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3850635446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040814c340 con 0x5604082422e0 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0d402640 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8dfeffd640 1 -- 172.21.15.3:0/1003122983 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8df80cf580 con 0x7f8e08104f20 2024-09-12T02:25:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009900 tx=0x7f8e04005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5ce0d86de98188 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8e0e404640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 0x7f8df40a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8e0806b400 tx=0x7f8e00009590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8df3fff640 1 -- 172.21.15.3:0/3850635446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04035030 con 0x5604082422e0 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8df3fff640 1 -- 172.21.15.3:0/3850635446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e040043c0 con 0x5604082422e0 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.557+0000 7f8df3fff640 1 -- 172.21.15.3:0/3850635446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040316b0 con 0x5604082422e0 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- 172.21.15.3:0/3850635446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 msgr2=0x560408246da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009900 tx=0x7f8e04005e50 comp rx=0 tx=0).stop 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- 172.21.15.3:0/3850635446 shutdown_connections 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082422e0 0x560408246da0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3850635446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082419a0 0x560408241da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- 172.21.15.3:0/3850635446 >> 172.21.15.3:0/3850635446 conn(0x56040823d010 msgr2=0x56040823d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- 172.21.15.3:0/3850635446 shutdown_connections 2024-09-12T02:25:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- 172.21.15.3:0/3850635446 wait complete. 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.558+0000 7f8e12bff740 1 -- start start 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 Processor -- start 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604082422e0 0x56040824c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560408247370 0x560408253180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56040824a2e0 con 0x5604082419a0 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x56040812e290 con 0x5604082422e0 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56040824a460 con 0x560408247370 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e12bff740 10 client.0 _ll_register_callbacks cb 0x56040815f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39120/0 (socket says 172.21.15.3:39120) 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3213420746 learned_addr learned my addr 172.21.15.3:0/3213420746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0d402640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604082422e0 0x56040824c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560408247370 msgr2=0x560408253180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.559+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560408247370 0x560408253180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3213420746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604082422e0 msgr2=0x56040824c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604082422e0 0x56040824c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e0dc03640 1 -- 172.21.15.3:0/3213420746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560407eb8210 con 0x5604082419a0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e0402aa00 tx=0x7f8e04031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a156634057c5232e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04052060 con 0x5604082419a0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8e04031d60 con 0x5604082419a0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e040043c0 con 0x5604082419a0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8e04035070 con 0x5604082419a0 2024-09-12T02:25:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e12bff740 10 client.16617 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560407f84620 con 0x5604082419a0 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e040043c0 con 0x5604082419a0 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f8e04004e60 con 0x5604082419a0 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e12bff740 10 client.16617 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.560+0000 7f8e12bff740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.561+0000 7f8e12bff740 20 client.16617 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.561+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560408257480 0x5604082598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.561+0000 7f8e12bff740 4 client.16617 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.561+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604081b2a20 con 0x560408257480 2024-09-12T02:25:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.561+0000 7f8e0d402640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560408257480 0x5604082598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.562+0000 7f8e0d402640 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560408257480 0x5604082598b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922a1f2cce236b7b server_cookie=65bbc840ffc2d663 in_seq=0 out_seq=0 2024-09-12T02:25:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.566+0000 7f8df17fa640 10 client.16617 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.567+0000 7f8df17fa640 1 -- 172.21.15.3:0/3213420746 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604081b2a20 con 0x560408257480 2024-09-12T02:25:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.567+0000 7f8df17fa640 10 client.16617 handle_command_reply: tid=0 2024-09-12T02:25:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.567+0000 7f8e12bff740 1 client.16617 shutdown 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.571+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 msgr2=0x56040824bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.572+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e0402aa00 tx=0x7f8e04031960 comp rx=0 tx=0).stop 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.572+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 shutdown_connections 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x560408257480 0x5604082598b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8df17fa640 0 client.16617 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560408247370 0x560408253180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5604082422e0 0x56040824c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8e12bff740 1 --2- 172.21.15.3:0/3213420746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5604082419a0 0x56040824bc00 secure :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e0402aa00 tx=0x7f8e04031960 comp rx=0 tx=0).stop 2024-09-12T02:25:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.573+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 >> 172.21.15.3:0/3213420746 conn(0x56040823d010 msgr2=0x56040823e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.576+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 shutdown_connections 2024-09-12T02:25:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.576+0000 7f8e12bff740 1 -- 172.21.15.3:0/3213420746 wait complete. 2024-09-12T02:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.582+0000 7f8e12bff740 20 client.16617 trim_cache size 0 max 16384 2024-09-12T02:25:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.615+0000 7f8e12bff740 1 Processor -- start 2024-09-12T02:25:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.615+0000 7f8e12bff740 1 -- start start 2024-09-12T02:25:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.615+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408241d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.615+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408253140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408241d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408241d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:45646/0 (socket says 172.21.15.3:45646) 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38798/0 (socket says 172.21.15.3:38798) 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0dc03640 1 -- 172.21.15.3:0/459018039 learned_addr learned my addr 172.21.15.3:0/459018039 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081a90f0 con 0x560408257480 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081aac30 con 0x560408247370 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081a97c0 con 0x5604082422e0 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.616+0000 7f8e0e404640 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408253140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 -- 172.21.15.3:0/459018039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 msgr2=0x560408241d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408241d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 -- 172.21.15.3:0/459018039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 msgr2=0x560408253140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408253140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 -- 172.21.15.3:0/459018039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56040823ce30 con 0x560408247370 2024-09-12T02:25:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.617+0000 7f8e0d402640 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8df80999e0 tx=0x7f8df8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e589776bb1e1e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.621+0000 7f8df37fe640 1 -- 172.21.15.3:0/459018039 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df809bd70 con 0x560408247370 2024-09-12T02:25:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.621+0000 7f8df37fe640 1 -- 172.21.15.3:0/459018039 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df80a0120 con 0x560408247370 2024-09-12T02:25:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.621+0000 7f8df37fe640 1 -- 172.21.15.3:0/459018039 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80cf110 con 0x560408247370 2024-09-12T02:25:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.622+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 msgr2=0x56040824c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.622+0000 7f8e12bff740 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8df80999e0 tx=0x7f8df8005880 comp rx=0 tx=0).stop 2024-09-12T02:25:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.624+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 shutdown_connections 2024-09-12T02:25:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.624+0000 7f8e12bff740 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408253140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.624+0000 7f8e12bff740 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x56040824c020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.624+0000 7f8e12bff740 1 --2- 172.21.15.3:0/459018039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408241d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.624+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 >> 172.21.15.3:0/459018039 conn(0x56040823d010 msgr2=0x56040823c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.626+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 shutdown_connections 2024-09-12T02:25:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.626+0000 7f8e12bff740 1 -- 172.21.15.3:0/459018039 wait complete. 2024-09-12T02:25:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.627+0000 7f8e12bff740 1 -- start start 2024-09-12T02:25:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.627+0000 7f8e12bff740 1 Processor -- start 2024-09-12T02:25:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e12bff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408251e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e0d402640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:38800/0 (socket says 172.21.15.3:38800) 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.628+0000 7f8e0dc03640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:45650/0 (socket says 172.21.15.3:45650) 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e0d402640 1 -- 172.21.15.3:0/1446228912 learned_addr learned my addr 172.21.15.3:0/1446228912 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081aa560 con 0x560408257480 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081a9e90 con 0x560408247370 2024-09-12T02:25:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081a70d0 con 0x5604082422e0 2024-09-12T02:25:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e12bff740 10 client.0 _ll_register_callbacks cb 0x7f8e04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.629+0000 7f8e0e404640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408251e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0dc03640 1 -- 172.21.15.3:0/1446228912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 msgr2=0x560408250e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0dc03640 1 -- 172.21.15.3:0/1446228912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 msgr2=0x560408251e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408251e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0dc03640 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560408252460 con 0x5604082422e0 2024-09-12T02:25:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.630+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:25:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.631+0000 7f8e0dc03640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005280 tx=0x7f8e04031ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4fa070a81e3d37c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.631+0000 7f8df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:25:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.631+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04052040 con 0x5604082422e0 2024-09-12T02:25:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.632+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df80a0120 con 0x5604082422e0 2024-09-12T02:25:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.631+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f8e040049d0 con 0x5604082422e0 2024-09-12T02:25:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.632+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8e04004bf0 con 0x5604082422e0 2024-09-12T02:25:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.634+0000 7f8e12bff740 10 client.26164 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.634+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5604082c2240 con 0x5604082422e0 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.636+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df80a0120 con 0x5604082422e0 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.636+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f8e04070060 con 0x5604082422e0 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 10 client.26164 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 20 client.26164 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5604082c2bd0 0x5604082c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 4 client.26164 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:25:42.638 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.637+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604081acd70 con 0x5604082c2bd0 2024-09-12T02:25:42.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.640+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5604082c2bd0 0x5604082c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.649+0000 7f8e0d402640 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5604082c2bd0 0x5604082c2fe0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484e2120e18ac74e server_cookie=5ea2b8e49fa808de in_seq=0 out_seq=0 2024-09-12T02:25:42.650 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:42.649+0000 7f8df17fa640 10 client.26164 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:44.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:43 smithi026 ceph-mon[35496]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 6.38k op/s 2024-09-12T02:25:44.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:43 smithi003 ceph-mon[31476]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 6.38k op/s 2024-09-12T02:25:44.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:43 smithi092 ceph-mon[36089]: pgmap v556: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 36 MiB/s wr, 6.38k op/s 2024-09-12T02:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.043+0000 7f8df17fa640 1 -- 172.21.15.3:0/1446228912 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604081acd70 con 0x5604082c2bd0 2024-09-12T02:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.043+0000 7f8df17fa640 10 client.26164 handle_command_reply: tid=0 2024-09-12T02:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 client.26164 shutdown 2024-09-12T02:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 msgr2=0x560408250900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005280 tx=0x7f8e04031ac0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 shutdown_connections 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5604082c2bd0 0x5604082c2fe0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560408257480 0x560408251e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560408247370 0x560408250e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 --2- 172.21.15.3:0/1446228912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5604082422e0 0x560408250900 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 >> 172.21.15.3:0/1446228912 conn(0x56040823d010 msgr2=0x56040823c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8df17fa640 0 client.26164 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 shutdown_connections 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.044+0000 7f8e12bff740 1 -- 172.21.15.3:0/1446228912 wait complete. 2024-09-12T02:25:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.045+0000 7f8e12bff740 20 client.26164 trim_cache size 0 max 16384 2024-09-12T02:25:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:25:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:25:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:25:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:25:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 msgr2=0x7f8df40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 0x7f8df40a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8e0806b400 tx=0x7f8e00009590 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 msgr2=0x7f8e081984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e9c0 tx=0x7f8df800ee80 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 shutdown_connections 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f8df40a2b80 0x7f8df40a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e08107850 0x7f8e0819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e08104f20 0x7f8e081984a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 --2- 172.21.15.3:0/1003122983 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e081025f0 0x7f8e08197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 >> 172.21.15.3:0/1003122983 conn(0x7f8e080fc1f0 msgr2=0x7f8e08102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 shutdown_connections 2024-09-12T02:25:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:44.046+0000 7f8e0fe67640 1 -- 172.21.15.3:0/1003122983 wait complete. 2024-09-12T02:25:46.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:45 smithi026 ceph-mon[35496]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 7.94k op/s 2024-09-12T02:25:46.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:45 smithi003 ceph-mon[31476]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 7.94k op/s 2024-09-12T02:25:46.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:45 smithi092 ceph-mon[36089]: pgmap v557: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 MiB/s wr, 7.94k op/s 2024-09-12T02:25:47.216 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db --set-id 76d698d7 --release --await' 2024-09-12T02:25:47.216 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db --set-id 76d698d7 --release --await 2024-09-12T02:25:47.711 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:25:48.014 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:47 smithi026 ceph-mon[35496]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 35 MiB/s wr, 6.28k op/s 2024-09-12T02:25:48.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:47 smithi003 ceph-mon[31476]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 35 MiB/s wr, 6.28k op/s 2024-09-12T02:25:48.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:47 smithi092 ceph-mon[36089]: pgmap v558: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 35 MiB/s wr, 6.28k op/s 2024-09-12T02:25:50.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:49 smithi026 ceph-mon[35496]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 35 MiB/s wr, 6.40k op/s 2024-09-12T02:25:50.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:49 smithi003 ceph-mon[31476]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 35 MiB/s wr, 6.40k op/s 2024-09-12T02:25:50.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:49 smithi092 ceph-mon[36089]: pgmap v559: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 35 MiB/s wr, 6.40k op/s 2024-09-12T02:25:52.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:51 smithi026 ceph-mon[35496]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 39 MiB/s wr, 7.21k op/s 2024-09-12T02:25:52.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:51 smithi003 ceph-mon[31476]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 39 MiB/s wr, 7.21k op/s 2024-09-12T02:25:52.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:51 smithi092 ceph-mon[36089]: pgmap v560: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 39 MiB/s wr, 7.21k op/s 2024-09-12T02:25:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 -- 172.21.15.3:0/553120173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc1036e0 msgr2=0x7f1ffc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 --2- 172.21.15.3:0/553120173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc107b90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009930 tx=0x7f1fe402a420 comp rx=0 tx=0).stop 2024-09-12T02:25:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 -- 172.21.15.3:0/553120173 shutdown_connections 2024-09-12T02:25:52.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 --2- 172.21.15.3:0/553120173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 0x7f1ffc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 --2- 172.21.15.3:0/553120173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc107b90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 --2- 172.21.15.3:0/553120173 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc102d30 0x7f1ffc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 -- 172.21.15.3:0/553120173 >> 172.21.15.3:0/553120173 conn(0x7f1ffc0fc7d0 msgr2=0x7f1ffc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 -- 172.21.15.3:0/553120173 shutdown_connections 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.052+0000 7f2000dcd640 1 -- 172.21.15.3:0/553120173 wait complete. 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.053+0000 7f2000dcd640 1 Processor -- start 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.053+0000 7f2000dcd640 1 -- start start 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 0x7f1ffc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1120a0 con 0x7f1ffc102d30 2024-09-12T02:25:52.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc111f20 con 0x7f1ffc1082a0 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f2000dcd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc112220 con 0x7f1ffc1036e0 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52870/0 (socket says 172.21.15.3:52870) 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 -- 172.21.15.3:0/1619655600 learned_addr learned my addr 172.21.15.3:0/1619655600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 0x7f1ffc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 -- 172.21.15.3:0/1619655600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc1036e0 msgr2=0x7f1ffc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 -- 172.21.15.3:0/1619655600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 msgr2=0x7f1ffc19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 0x7f1ffc19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 -- 172.21.15.3:0/1619655600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc19d570 con 0x7f1ffc102d30 2024-09-12T02:25:52.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.054+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.055+0000 7f1ff37fe640 1 -- 172.21.15.3:0/1619655600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00cd80 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.055+0000 7f1ff37fe640 1 -- 172.21.15.3:0/1619655600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec005500 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.055+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc19d860 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.055+0000 7f1ff37fe640 1 -- 172.21.15.3:0/1619655600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec010690 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.055+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1a5140 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.057+0000 7f1ff37fe640 1 -- 172.21.15.3:0/1619655600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1fec0046c0 con 0x7f1ffc102d30 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.057+0000 7f20031a6740 1 Processor -- start 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 -- start start 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654739168a0 0x565473925110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473927a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56547387b620 con 0x565473915f80 2024-09-12T02:25:52.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565473822e50 con 0x5654739168a0 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f20031a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5654738775e0 con 0x565473925650 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52884/0 (socket says 172.21.15.3:52884) 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2093762342 learned_addr learned my addr 172.21.15.3:0/2093762342 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654739168a0 0x565473925110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473927a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ff37fe640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 0x7f1fe80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2093762342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 msgr2=0x565473927a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473927a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2093762342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654739168a0 msgr2=0x565473925110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654739168a0 0x565473925110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.058+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2093762342 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565473813d40 con 0x565473915f80 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ff37fe640 1 -- 172.21.15.3:0/1619655600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1fec0cf430 con 0x7f1ffc102d30 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009f70 tx=0x7f1ff4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bfeb78a8fb1018 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 0x7f1fe80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ff0ff9640 1 -- 172.21.15.3:0/2093762342 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4022070 con 0x565473915f80 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ff0ff9640 1 -- 172.21.15.3:0/2093762342 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff40113e0 con 0x565473915f80 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.059+0000 7f1ff0ff9640 1 -- 172.21.15.3:0/2093762342 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4016070 con 0x565473915f80 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- 172.21.15.3:0/2093762342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 msgr2=0x565473916360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009f70 tx=0x7f1ff4010800 comp rx=0 tx=0).stop 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 0x7f1fe80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401f000 tx=0x7f1ff4022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- 172.21.15.3:0/2093762342 shutdown_connections 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473927a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654739168a0 0x565473925110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 --2- 172.21.15.3:0/2093762342 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473915f80 0x565473916360 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- 172.21.15.3:0/2093762342 >> 172.21.15.3:0/2093762342 conn(0x5654739125f0 msgr2=0x5654739129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- 172.21.15.3:0/2093762342 shutdown_connections 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- 172.21.15.3:0/2093762342 wait complete. 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 -- start start 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.060+0000 7f20031a6740 1 Processor -- start 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x5654739225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473923070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5654739281c0 con 0x5654739168a0 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x565473803f10 con 0x565473915f80 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x565473928340 con 0x565473925650 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f20031a6740 10 client.0 _ll_register_callbacks cb 0x565473827db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:52.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52896/0 (socket says 172.21.15.3:52896) 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 -- 172.21.15.3:0/962637131 learned_addr learned my addr 172.21.15.3:0/962637131 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x5654739225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473923070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 -- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 msgr2=0x565473923070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473923070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 -- 172.21.15.3:0/962637131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 msgr2=0x5654739225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x5654739225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ff3fff640 1 -- 172.21.15.3:0/962637131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547358a580 con 0x5654739168a0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.061+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x5654739225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4000c00 tx=0x7f1fe4032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52becb9f8d5ad614 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:52.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4035040 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1fe4031970 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff40113e0 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1fe4004040 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 10 client.16635 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565473654960 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff40113e0 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1fe4004890 con 0x5654739168a0 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 10 client.16635 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 10 client.16635 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 20 client.16635 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565473935240 0x565473937650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 4 client.16635 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:52.593 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.062+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565473888b90 con 0x565473935240 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.063+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565473935240 0x565473937650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.063+0000 7f1fe1ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.063+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565473935240 0x565473937650 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4eb66f621be5b77 server_cookie=20ccc6eb44fc65df in_seq=0 out_seq=0 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/962637131 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565473888b90 con 0x565473935240 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f1fe1ffb640 10 client.16635 handle_command_reply: tid=0 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 client.16635 shutdown 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 msgr2=0x565473922b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4000c00 tx=0x7f1fe4032ed0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 shutdown_connections 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x565473935240 0x565473937650 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x565473923070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x565473922b30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f1fe1ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 --2- 172.21.15.3:0/962637131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x5654739225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 >> 172.21.15.3:0/962637131 conn(0x5654739125f0 msgr2=0x565473918260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 shutdown_connections 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 1 -- 172.21.15.3:0/962637131 wait complete. 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.064+0000 7f20031a6740 20 client.16635 trim_cache size 0 max 16384 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 Processor -- start 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 -- start start 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x56547393cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x56547391f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ab4a0 con 0x5654739168a0 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1acfe0 con 0x565473915f80 2024-09-12T02:25:52.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f20031a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1abb70 con 0x565473925650 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52904/0 (socket says 172.21.15.3:52904) 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 -- 172.21.15.3:0/1510299958 learned_addr learned my addr 172.21.15.3:0/1510299958 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x56547391f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x56547393cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 -- 172.21.15.3:0/1510299958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 msgr2=0x56547391f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x56547391f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 -- 172.21.15.3:0/1510299958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 msgr2=0x56547393cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x56547393cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 -- 172.21.15.3:0/1510299958 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565473937cd0 con 0x5654739168a0 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x56547391f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.082+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006010 tx=0x7f1fe4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1df2e199f1e2a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f1fe3fff640 1 -- 172.21.15.3:0/1510299958 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4035030 con 0x5654739168a0 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f1fe3fff640 1 -- 172.21.15.3:0/1510299958 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4031980 con 0x5654739168a0 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f1fe3fff640 1 -- 172.21.15.3:0/1510299958 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40044e0 con 0x5654739168a0 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 -- 172.21.15.3:0/1510299958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 msgr2=0x56547393d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006010 tx=0x7f1fe4001e70 comp rx=0 tx=0).stop 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 -- 172.21.15.3:0/1510299958 shutdown_connections 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x565473925650 0x56547391f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5654739168a0 0x56547393d460 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 --2- 172.21.15.3:0/1510299958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x56547393cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 -- 172.21.15.3:0/1510299958 >> 172.21.15.3:0/1510299958 conn(0x5654739125f0 msgr2=0x565473920ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 -- 172.21.15.3:0/1510299958 shutdown_connections 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.083+0000 7f20031a6740 1 -- 172.21.15.3:0/1510299958 wait complete. 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 -- start start 2024-09-12T02:25:52.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 Processor -- start 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x565473929b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5654739168a0 0x56547392a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ac910 con 0x565473925650 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1ac240 con 0x565473915f80 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc1a9480 con 0x5654739168a0 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f20031a6740 10 client.0 _ll_register_callbacks cb 0x7f1fe4005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52910/0 (socket says 172.21.15.3:52910) 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2333236469 learned_addr learned my addr 172.21.15.3:0/2333236469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x565473929b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.084+0000 7f1ff3fff640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5654739168a0 0x56547392a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5654739168a0 msgr2=0x56547392a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5654739168a0 0x56547392a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2333236469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 msgr2=0x565473929b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x565473929b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 -- 172.21.15.3:0/2333236469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547399f530 con 0x565473925650 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1ffad9d640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009ec0 tx=0x7f1ff40129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab359c15ec85040 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4016040 con 0x565473925650 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1ff401cdf0 con 0x565473925650 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe4031980 con 0x565473925650 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ff4009bb0 con 0x565473925650 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f20031a6740 10 client.16641 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:52.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5654739a07c0 con 0x565473925650 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.085+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4031980 con 0x565473925650 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f1ff400f040 con 0x565473925650 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 10 client.16641 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 10 client.16641 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 20 client.16641 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5654739a0bf0 0x5654739a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 4 client.16641 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "76d698d7", "release": true, "await": true}] 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "set_id": "76d698d7", "release": true, "await": true}) -- 0x5654738830a0 con 0x5654739a0bf0 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5654739a0bf0 0x5654739a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f1ffa59c640 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5654739a0bf0 0x5654739a3010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c6393d5299c7d2 server_cookie=6eb7d44912c671ea in_seq=0 out_seq=0 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.086+0000 7f1fe1ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f1fe1ffb640 1 -- 172.21.15.3:0/2333236469 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5654738830a0 con 0x5654739a0bf0 2024-09-12T02:25:52.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f1fe1ffb640 10 client.16641 handle_command_reply: tid=0 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 client.16641 shutdown 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 msgr2=0x56547399eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009ec0 tx=0x7f1ff40129e0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 shutdown_connections 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5654739a0bf0 0x5654739a3010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f1fe1ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x565473925650 0x56547399eef0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:25:52.601 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 37, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 16, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "76d698d7": { 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 16, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 140.2, 2024-09-12T02:25:52.602 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:25:52.603 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5654739168a0 0x56547392a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 --2- 172.21.15.3:0/2333236469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x565473915f80 0x565473929b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 >> 172.21.15.3:0/2333236469 conn(0x5654739125f0 msgr2=0x565473922530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 shutdown_connections 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 1 -- 172.21.15.3:0/2333236469 wait complete. 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.113+0000 7f20031a6740 20 client.16641 trim_cache size 0 max 16384 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 msgr2=0x7f1fe80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 0x7f1fe80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401f000 tx=0x7f1ff4022000 comp rx=0 tx=0).stop 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 msgr2=0x7f1ffc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00edf0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 shutdown_connections 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f1fe80a2b80 0x7f1fe80a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ffc1082a0 0x7f1ffc19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ffc1036e0 0x7f1ffc198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 --2- 172.21.15.3:0/1619655600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1ffc102d30 0x7f1ffc1985c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.114+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 >> 172.21.15.3:0/1619655600 conn(0x7f1ffc0fc7d0 msgr2=0x7f1ffc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.115+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 shutdown_connections 2024-09-12T02:25:52.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:25:51.115+0000 7f2000dcd640 1 -- 172.21.15.3:0/1619655600 wait complete. 2024-09-12T02:25:54.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:53 smithi026 ceph-mon[35496]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 33 MiB/s wr, 6.16k op/s 2024-09-12T02:25:54.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:53 smithi003 ceph-mon[31476]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 33 MiB/s wr, 6.16k op/s 2024-09-12T02:25:54.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:53 smithi092 ceph-mon[36089]: pgmap v561: 97 pgs: 97 active+clean; 13 GiB data, 39 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 33 MiB/s wr, 6.16k op/s 2024-09-12T02:25:55.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:54 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:25:55.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:54 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:25:55.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:54 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 8.62k op/s 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: Unable to set osd_memory_target on smithi092 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-12T02:25:56.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-12T02:25:56.016 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:55 smithi026 ceph-mon[35496]: Unable to set osd_memory_target on smithi026 to 185698508: error parsing value: Value '185698508' is below minimum 939524096 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 8.62k op/s 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: Unable to set osd_memory_target on smithi092 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-12T02:25:56.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:55 smithi003 ceph-mon[31476]: Unable to set osd_memory_target on smithi026 to 185698508: error parsing value: Value '185698508' is below minimum 939524096 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: pgmap v562: 97 pgs: 97 active+clean; 13 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 8.62k op/s 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi092 to 176.0M 2024-09-12T02:25:56.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: Unable to set osd_memory_target on smithi092 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: Adjusting osd_memory_target on smithi026 to 177.0M 2024-09-12T02:25:56.040 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:55 smithi092 ceph-mon[36089]: Unable to set osd_memory_target on smithi026 to 185698508: error parsing value: Value '185698508' is below minimum 939524096 2024-09-12T02:25:57.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:25:57.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:25:57.015 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:25:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:25:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:25:57.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:25:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:25:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:25:57.039 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:25:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.218+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/2205149496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00736b0 msgr2=0x7fa9e0119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.218+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/2205149496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00736b0 0x7fa9e0119130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa9d001a910 tx=0x7fa9d003b470 comp rx=0 tx=0).stop 2024-09-12T02:25:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.219+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/2205149496 shutdown_connections 2024-09-12T02:25:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.219+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/2205149496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e00736b0 0x7fa9e0119130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.219+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/2205149496 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 0x7fa9e0073170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.219+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/2205149496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e0109700 0x7fa9e0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.219+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/2205149496 >> 172.21.15.3:0/2205149496 conn(0x7fa9e006ecd0 msgr2=0x7fa9e0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.225+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/2205149496 shutdown_connections 2024-09-12T02:25:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.225+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/2205149496 wait complete. 2024-09-12T02:25:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.226+0000 7fa9e5fd8640 1 Processor -- start 2024-09-12T02:25:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.226+0000 7fa9e5fd8640 1 -- start start 2024-09-12T02:25:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.226+0000 7fa9e5fd8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 0x7fa9e019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.227+0000 7fa9e5fd8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.228+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.232+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:48050/0 (socket says 172.21.15.3:48050) 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.232+0000 7fa9e5fd8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.232+0000 7fa9e5fd8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01135e0 con 0x7fa9e0072d10 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.232+0000 7fa9e5fd8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0113460 con 0x7fa9e00736b0 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.233+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.233+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:34662/0 (socket says 172.21.15.3:34662) 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.233+0000 7fa9dffff640 1 -- 172.21.15.3:0/314561631 learned_addr learned my addr 172.21.15.3:0/314561631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.235+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0113760 con 0x7fa9e0109700 2024-09-12T02:25:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.236+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 0x7fa9e019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9dffff640 1 -- 172.21.15.3:0/314561631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 msgr2=0x7fa9e019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9dffff640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9dffff640 1 -- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 msgr2=0x7fa9e019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9dffff640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 0x7fa9e019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9dffff640 1 -- 172.21.15.3:0/314561631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e01a60b0 con 0x7fa9e0109700 2024-09-12T02:25:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.241+0000 7fa9deffd640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:25:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.242+0000 7fa9dffff640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005ab0 tx=0x7fa9d0004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.243+0000 7fa9dcff9640 1 -- 172.21.15.3:0/314561631 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d004f020 con 0x7fa9e0109700 2024-09-12T02:25:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.243+0000 7fa9dcff9640 1 -- 172.21.15.3:0/314561631 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d0004c20 con 0x7fa9e0109700 2024-09-12T02:25:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.243+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e01a6340 con 0x7fa9e0109700 2024-09-12T02:25:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.243+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e01a68b0 con 0x7fa9e0109700 2024-09-12T02:25:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.248+0000 7fa9e8e4b740 1 Processor -- start 2024-09-12T02:25:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.248+0000 7fa9dcff9640 1 -- 172.21.15.3:0/314561631 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0046070 con 0x7fa9e0109700 2024-09-12T02:25:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 -- start start 2024-09-12T02:25:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f994543da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f994549370 0x55f99454b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f9944a72f0 con 0x55f9945439a0 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f99444f270 con 0x55f994549370 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.250+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f9944a32b0 con 0x55f9945442e0 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.252+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.252+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:34666/0 (socket says 172.21.15.3:34666) 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.252+0000 7fa9df7fe640 1 -- 172.21.15.3:0/1523739861 learned_addr learned my addr 172.21.15.3:0/1523739861 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 -- 172.21.15.3:0/1523739861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f994549370 msgr2=0x55f99454b830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f994549370 0x55f99454b830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 -- 172.21.15.3:0/1523739861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 msgr2=0x55f994543da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:25:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f994543da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 -- 172.21.15.3:0/1523739861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99444e340 con 0x55f9945442e0 2024-09-12T02:25:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.253+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a40 tx=0x7fa9d4009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3542d8a91a31d5c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.258+0000 7fa9c5ffb640 1 -- 172.21.15.3:0/1523739861 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4017070 con 0x55f9945442e0 2024-09-12T02:25:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.258+0000 7fa9c5ffb640 1 -- 172.21.15.3:0/1523739861 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4013470 con 0x55f9945442e0 2024-09-12T02:25:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.259+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/1523739861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 msgr2=0x55f994548da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.259+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009a40 tx=0x7fa9d4009f00 comp rx=0 tx=0).stop 2024-09-12T02:25:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.259+0000 7fa9dcff9640 1 -- 172.21.15.3:0/314561631 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa9d0052020 con 0x7fa9e0109700 2024-09-12T02:25:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.260+0000 7fa9dcff9640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.260+0000 7fa9dcff9640 1 -- 172.21.15.3:0/314561631 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9d0100090 con 0x7fa9e0109700 2024-09-12T02:25:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.262+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/1523739861 shutdown_connections 2024-09-12T02:25:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.262+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f994549370 0x55f99454b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.262+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994548da0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.262+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/1523739861 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f994543da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.262+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/1523739861 >> 172.21.15.3:0/1523739861 conn(0x55f99453f010 msgr2=0x55f99453f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.264+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/1523739861 shutdown_connections 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.264+0000 7fa9deffd640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.265+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/1523739861 wait complete. 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.265+0000 7fa9e8e4b740 1 -- start start 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 Processor -- start 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945439a0 0x55f99454dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f99454e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f99454c390 con 0x55f994549370 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f994430290 con 0x55f9945442e0 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f99454c510 con 0x55f9945439a0 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9e8e4b740 10 client.0 _ll_register_callbacks cb 0x55f994461140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52950/0 (socket says 172.21.15.3:52950) 2024-09-12T02:25:57.270 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.266+0000 7fa9dffff640 1 -- 172.21.15.3:0/741964577 learned_addr learned my addr 172.21.15.3:0/741964577 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.269+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945439a0 0x55f99454dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.269+0000 7fa9deffd640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f99454e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.272+0000 7fa9dffff640 1 -- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945439a0 msgr2=0x55f99454dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.272+0000 7fa9dffff640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945439a0 0x55f99454dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.272+0000 7fa9dffff640 1 -- 172.21.15.3:0/741964577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 msgr2=0x55f99454e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.272+0000 7fa9dffff640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f99454e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.272+0000 7fa9dffff640 1 -- 172.21.15.3:0/741964577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9941ba210 con 0x55f994549370 2024-09-12T02:25:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.273+0000 7fa9deffd640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9e019f9a0 tx=0x7fa9d8009460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.274+0000 7fa9dffff640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00c9250 tx=0x7fa9d0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2394827b47ecc1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.277+0000 7fa9beffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.277+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00d0c30 con 0x55f994549370 2024-09-12T02:25:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.277+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9d00b7070 con 0x55f994549370 2024-09-12T02:25:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.278+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d4013470 con 0x55f994549370 2024-09-12T02:25:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.282+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa9d00d54f0 con 0x55f994549370 2024-09-12T02:25:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.284+0000 7fa9e8e4b740 10 client.16650 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.284+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f994286620 con 0x55f994549370 2024-09-12T02:25:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4013470 con 0x55f994549370 2024-09-12T02:25:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa9d00d5060 con 0x55f994549370 2024-09-12T02:25:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 10 client.16650 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 20 client.16650 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f99457b940 0x55f99457ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 4 client.16650 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:25:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9944b4a20 con 0x55f99457b940 2024-09-12T02:25:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.290+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f99457b940 0x55f99457ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.295+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f99457b940 0x55f99457ddc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59007014f37c86f6 server_cookie=8216ed88d62fd3f0 in_seq=0 out_seq=0 2024-09-12T02:25:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.295+0000 7fa9beffd640 10 client.16650 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.299+0000 7fa9beffd640 1 -- 172.21.15.3:0/741964577 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9944b4a20 con 0x55f99457b940 2024-09-12T02:25:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.299+0000 7fa9beffd640 10 client.16650 handle_command_reply: tid=0 2024-09-12T02:25:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.299+0000 7fa9e8e4b740 1 client.16650 shutdown 2024-09-12T02:25:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.311+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 msgr2=0x55f994555120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.311+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00c9250 tx=0x7fa9d0005590 comp rx=0 tx=0).stop 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 shutdown_connections 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f99457b940 0x55f99457ddc0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994555120 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f99454e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/741964577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945439a0 0x55f99454dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 >> 172.21.15.3:0/741964577 conn(0x55f99453f010 msgr2=0x55f994540820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.315+0000 7fa9beffd640 0 client.16650 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.320+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 shutdown_connections 2024-09-12T02:25:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.323+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/741964577 wait complete. 2024-09-12T02:25:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.326+0000 7fa9e8e4b740 20 client.16650 trim_cache size 0 max 16384 2024-09-12T02:25:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.361+0000 7fa9e8e4b740 1 Processor -- start 2024-09-12T02:25:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.361+0000 7fa9e8e4b740 1 -- start start 2024-09-12T02:25:57.363 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f99453e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f9945550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 0x55f9945826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc00e570 con 0x55f9945439a0 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc0100b0 con 0x55f9945442e0 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 0x55f9945826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 0x55f9945826c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:34686/0 (socket says 172.21.15.3:34686) 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.362+0000 7fa9dffff640 1 -- 172.21.15.3:0/3185491204 learned_addr learned my addr 172.21.15.3:0/3185491204 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.363+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc00ec40 con 0x55f994549370 2024-09-12T02:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.363+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f99453e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.363+0000 7fa9deffd640 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f9945550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 -- 172.21.15.3:0/3185491204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 msgr2=0x55f9945826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 0x55f9945826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.365 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 -- 172.21.15.3:0/3185491204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 msgr2=0x55f9945550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f9945550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 -- 172.21.15.3:0/3185491204 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99454d6e0 con 0x55f9945439a0 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.364+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f99453e0b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4006a00 tx=0x7fa9d40090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd204a12b145d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.365+0000 7fa9c57fa640 1 -- 172.21.15.3:0/3185491204 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4029070 con 0x55f9945439a0 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.365+0000 7fa9c57fa640 1 -- 172.21.15.3:0/3185491204 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4013950 con 0x55f9945439a0 2024-09-12T02:25:57.366 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.365+0000 7fa9c57fa640 1 -- 172.21.15.3:0/3185491204 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4017050 con 0x55f9945439a0 2024-09-12T02:25:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.366+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 msgr2=0x55f99453e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:57.367 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.366+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f99453e0b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4006a00 tx=0x7fa9d40090e0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.371+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 shutdown_connections 2024-09-12T02:25:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.371+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f994549370 0x55f9945826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.371+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945442e0 0x55f9945550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.371+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/3185491204 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f9945439a0 0x55f99453e0b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.371+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 >> 172.21.15.3:0/3185491204 conn(0x55f99453f010 msgr2=0x55f99453d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.374+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 shutdown_connections 2024-09-12T02:25:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.374+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/3185491204 wait complete. 2024-09-12T02:25:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.375+0000 7fa9e8e4b740 1 -- start start 2024-09-12T02:25:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.375+0000 7fa9e8e4b740 1 Processor -- start 2024-09-12T02:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945439a0 0x55f994552e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994553340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9e8e4b740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994554350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9e8e4b740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc00f9e0 con 0x55f994549370 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994553340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9df7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945439a0 0x55f994552e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994554350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994553340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:34698/0 (socket says 172.21.15.3:34698) 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.376+0000 7fa9deffd640 1 -- 172.21.15.3:0/2081748765 learned_addr learned my addr 172.21.15.3:0/2081748765 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.377+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc00f310 con 0x55f9945439a0 2024-09-12T02:25:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.377+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc00c550 con 0x55f9945442e0 2024-09-12T02:25:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.377+0000 7fa9e8e4b740 10 client.0 _ll_register_callbacks cb 0x7fa9d00cc650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:25:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.379+0000 7fa9dffff640 1 -- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 msgr2=0x55f994553340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.379+0000 7fa9dffff640 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994553340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.379+0000 7fa9dffff640 1 -- 172.21.15.3:0/2081748765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945439a0 msgr2=0x55f994552e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:25:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.379+0000 7fa9dffff640 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945439a0 0x55f994552e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.379+0000 7fa9dffff640 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f99453a2d0 con 0x55f994549370 2024-09-12T02:25:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.380+0000 7fa9dffff640 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994554350 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00d4fc0 tx=0x7fa9d00cc290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0337a436674f601 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:25:57.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.382+0000 7fa9beffd640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.382+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0096060 con 0x55f994549370 2024-09-12T02:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.383+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d4013950 con 0x55f994549370 2024-09-12T02:25:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.383+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fa9d00b70b0 con 0x55f994549370 2024-09-12T02:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.384+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa9d00cc3c0 con 0x55f994549370 2024-09-12T02:25:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.384+0000 7fa9e8e4b740 10 client.16656 fetch_fsmap learned FSMap version 37 2024-09-12T02:25:57.386 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.385+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f9945e6b00 con 0x55f994549370 2024-09-12T02:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4013950 con 0x55f994549370 2024-09-12T02:25:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fa9d00cbcd0 con 0x55f994549370 2024-09-12T02:25:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 10 client.16656 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T02:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 20 client.16656 populate_metadata read hostname 'smithi003' 2024-09-12T02:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f9945e7270 0x55f9945e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 4 client.16656 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:25:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.388+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9944aed70 con 0x55f9945e7270 2024-09-12T02:25:57.396 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.395+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f9945e7270 0x55f9945e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.401+0000 7fa9df7fe640 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f9945e7270 0x55f9945e9690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfbf202c3ad454a server_cookie=5e694a45ab72b1ad in_seq=0 out_seq=0 2024-09-12T02:25:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:57.401+0000 7fa9beffd640 10 client.16656 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:58.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:57 smithi026 ceph-mon[35496]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 7.65k op/s 2024-09-12T02:25:58.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:57 smithi003 ceph-mon[31476]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 7.65k op/s 2024-09-12T02:25:58.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:57 smithi092 ceph-mon[36089]: pgmap v563: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 39 MiB/s wr, 7.65k op/s 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.643+0000 7fa9beffd640 1 -- 172.21.15.3:0/2081748765 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9944aed70 con 0x55f9945e7270 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.643+0000 7fa9beffd640 10 client.16656 handle_command_reply: tid=0 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.643+0000 7fa9e8e4b740 1 client.16656 shutdown 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.643+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 msgr2=0x55f994554350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.643+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994554350 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00d4fc0 tx=0x7fa9d00cc290 comp rx=0 tx=0).stop 2024-09-12T02:25:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 shutdown_connections 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f9945e7270 0x55f9945e9690 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f994549370 0x55f994554350 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f9945442e0 0x55f994553340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 --2- 172.21.15.3:0/2081748765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9945439a0 0x55f994552e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 >> 172.21.15.3:0/2081748765 conn(0x55f99453f010 msgr2=0x55f99453e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9beffd640 0 client.16656 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 shutdown_connections 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 1 -- 172.21.15.3:0/2081748765 wait complete. 2024-09-12T02:25:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.644+0000 7fa9e8e4b740 20 client.16656 trim_cache size 0 max 16384 2024-09-12T02:25:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:25:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:25:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:25:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 msgr2=0x7fa9c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9e019f9a0 tx=0x7fa9d8009460 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 msgr2=0x7fa9e01a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005ab0 tx=0x7fa9d0004a30 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 shutdown_connections 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9e0109700 0x7fa9e01a5a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9e00736b0 0x7fa9e019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 --2- 172.21.15.3:0/314561631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa9e0072d10 0x7fa9e019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:25:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 >> 172.21.15.3:0/314561631 conn(0x7fa9e006ecd0 msgr2=0x7fa9e0116160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:25:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.645+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 shutdown_connections 2024-09-12T02:25:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:25:58.646+0000 7fa9e5fd8640 1 -- 172.21.15.3:0/314561631 wait complete. 2024-09-12T02:26:00.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:25:59 smithi026 ceph-mon[35496]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 7.18k op/s 2024-09-12T02:26:00.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:25:59 smithi003 ceph-mon[31476]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 7.18k op/s 2024-09-12T02:26:00.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:25:59 smithi092 ceph-mon[36089]: pgmap v564: 97 pgs: 97 active+clean; 13 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 7.18k op/s 2024-09-12T02:26:01.402 INFO:tasks.workunit.client.0.smithi003.stdout:Syncing()...10 sec 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:FFSB benchmark finished at: Thu Sep 12 02:26:01 2024 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:Results: 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:Benchmark took 344.53 sec 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:Total Results 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:=============== 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: ======= ============ ========= ======= =========== ========== 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: write : 11185920 32467.27 100.000% 100.000% 127MB/sec 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:- 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout:32467.27 Transactions per Second 2024-09-12T02:26:01.403 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.404 INFO:tasks.workunit.client.0.smithi003.stdout:Throughput Results 2024-09-12T02:26:01.404 INFO:tasks.workunit.client.0.smithi003.stdout:=================== 2024-09-12T02:26:01.404 INFO:tasks.workunit.client.0.smithi003.stdout:Write Throughput: 127MB/sec 2024-09-12T02:26:01.456 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout:System Call Latency statistics in millisecs 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout:===== 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: Min Avg Max Total Calls 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: ======== ======== ======== ============ 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout:[ open] 0.005000 0.528446 488.496002 8739 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T02:26:01.457 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[4] 200.000000 - 500.000000 : 5 2024-09-12T02:26:01.458 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[5] 100.000000 - 200.000000 : 4 2024-09-12T02:26:01.458 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[6] 50.000000 - 100.000000 : 4 2024-09-12T02:26:01.458 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[7] 20.000000 - 50.000000 : 18 2024-09-12T02:26:01.458 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[8] 10.000000 - 20.000000 : 53 2024-09-12T02:26:01.458 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[9] 5.000000 - 10.000000 : 54 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[10] 2.000000 - 5.000000 : 72 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[11] 1.000000 - 2.000000 : 56 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[12] 0.500000 - 1.000000 : 29 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[14] 0.100000 - 0.200000 : 6 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[15] 0.050000 - 0.100000 : 52 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[16] 0.020000 - 0.050000 : 3591 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[17] 0.010000 - 0.020000 : 3830 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[18] 0.000000 - 0.010000 : 963 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.459 INFO:tasks.workunit.client.0.smithi003.stdout:[ write] 0.001000 0.954314 56266.144531 11185920 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[0] 5000.000000 - 10000.000000 : 210 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[1] 2000.000000 - 5000.000000 : 348 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[2] 1000.000000 - 2000.000000 : 318 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[3] 500.000000 - 1000.000000 : 409 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[4] 200.000000 - 500.000000 : 765 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[5] 100.000000 - 200.000000 : 2323 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[6] 50.000000 - 100.000000 : 7750 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[7] 20.000000 - 50.000000 : 6039 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[8] 10.000000 - 20.000000 : 2346 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[9] 5.000000 - 10.000000 : 1791 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[10] 2.000000 - 5.000000 : 1464 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[11] 1.000000 - 2.000000 : 1033 2024-09-12T02:26:01.460 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[12] 0.500000 - 1.000000 : 958 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[13] 0.200000 - 0.500000 : 1064 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[14] 0.100000 - 0.200000 : 1351 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[15] 0.050000 - 0.100000 : 4081 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[16] 0.020000 - 0.050000 : 26835 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[17] 0.010000 - 0.020000 : 86950 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[18] 0.000000 - 0.010000 : 11039654 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout:[ lseek] 0.000000 0.001043 53.334000 11185920 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T02:26:01.461 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[6] 50.000000 - 100.000000 : 1 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[7] 20.000000 - 50.000000 : 77 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[8] 10.000000 - 20.000000 : 103 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[9] 5.000000 - 10.000000 : 65 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[10] 2.000000 - 5.000000 : 105 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[11] 1.000000 - 2.000000 : 93 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[12] 0.500000 - 1.000000 : 102 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[13] 0.200000 - 0.500000 : 122 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[14] 0.100000 - 0.200000 : 124 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[15] 0.050000 - 0.100000 : 347 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[16] 0.020000 - 0.050000 : 1940 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[17] 0.010000 - 0.020000 : 4811 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[18] 0.000000 - 0.010000 : 11178030 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.462 INFO:tasks.workunit.client.0.smithi003.stdout:[ close] 0.001000 0.004908 0.502000 8739 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[9] 5.000000 - 10.000000 : 0 2024-09-12T02:26:01.463 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[10] 2.000000 - 5.000000 : 0 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[11] 1.000000 - 2.000000 : 0 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[12] 0.500000 - 1.000000 : 1 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[13] 0.200000 - 0.500000 : 2 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[14] 0.100000 - 0.200000 : 1 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[15] 0.050000 - 0.100000 : 2 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[16] 0.020000 - 0.050000 : 18 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[17] 0.010000 - 0.020000 : 44 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: msec_range[18] 0.000000 - 0.010000 : 8671 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.464 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.465 INFO:tasks.workunit.client.0.smithi003.stdout: 2024-09-12T02:26:01.465 INFO:tasks.workunit.client.0.smithi003.stdout:2.9% User Time 2024-09-12T02:26:01.465 INFO:tasks.workunit.client.0.smithi003.stdout:10.9% System Time 2024-09-12T02:26:01.465 INFO:tasks.workunit.client.0.smithi003.stdout:13.8% CPU Utilization 2024-09-12T02:26:01.468 INFO:tasks.workunit.client.0.smithi003.stderr:+ cd .. 2024-09-12T02:26:01.472 INFO:tasks.workunit.client.0.smithi003.stderr:+ rm -r tmp ffsb 2024-09-12T02:26:02.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:01 smithi026 ceph-mon[35496]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 9.11k op/s 2024-09-12T02:26:02.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:01 smithi003 ceph-mon[31476]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 9.11k op/s 2024-09-12T02:26:02.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:01 smithi092 ceph-mon[36089]: pgmap v565: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 MiB/s wr, 9.11k op/s 2024-09-12T02:26:02.974 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T02:26:02.974 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T02:26:03.030 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-12T02:26:03.030 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T02:26:03.816 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '76d698d7', total seconds elapsed: 85.1 2024-09-12T02:26:03.817 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T02:26:03.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:26:04.028 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:03 smithi003 ceph-mon[31476]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 33 MiB/s wr, 7.16k op/s 2024-09-12T02:26:04.029 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:03 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:04.078 DEBUG:teuthology.parallel:result is None 2024-09-12T02:26:04.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:26:04.154 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T02:26:04.154 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T02:26:04.165 INFO:tasks.quiescer:joining Quiescers 2024-09-12T02:26:04.165 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T02:26:04.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:26:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:03 smithi026 ceph-mon[35496]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 33 MiB/s wr, 7.16k op/s 2024-09-12T02:26:04.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:04 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:04.273 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:04.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:03 smithi092 ceph-mon[36089]: pgmap v566: 97 pgs: 97 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 33 MiB/s wr, 7.16k op/s 2024-09-12T02:26:04.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:03 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:06.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:05 smithi026 ceph-mon[35496]: pgmap v567: 97 pgs: 97 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-09-12T02:26:06.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:05 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:05 smithi003 ceph-mon[31476]: pgmap v567: 97 pgs: 97 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-09-12T02:26:06.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:05 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:05 smithi092 ceph-mon[36089]: pgmap v567: 97 pgs: 97 active+clean; 12 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 32 MiB/s wr, 7.06k op/s 2024-09-12T02:26:06.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:05 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:07.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 -- 172.21.15.3:0/1634971794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 msgr2=0x7f131806bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 --2- 172.21.15.3:0/1634971794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f131806bc80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1314009890 tx=0x7f131402a380 comp rx=0 tx=0).stop 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 -- 172.21.15.3:0/1634971794 shutdown_connections 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 --2- 172.21.15.3:0/1634971794 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f131811d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 --2- 172.21.15.3:0/1634971794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 0x7f131811ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 --2- 172.21.15.3:0/1634971794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f131806bc80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 -- 172.21.15.3:0/1634971794 >> 172.21.15.3:0/1634971794 conn(0x7f1318071390 msgr2=0x7f13180737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 -- 172.21.15.3:0/1634971794 shutdown_connections 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.692+0000 7f131ed64640 1 -- 172.21.15.3:0/1634971794 wait complete. 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.693+0000 7f131ed64640 1 Processor -- start 2024-09-12T02:26:07.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 -- start start 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f13181a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 0x7f13181a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1318120c20 con 0x7f131811b380 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1318120aa0 con 0x7f131806b8a0 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.694+0000 7f131ed64640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1318120da0 con 0x7f131806c1c0 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131d561640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 0x7f13181a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:07.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53540/0 (socket says 172.21.15.3:53540) 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 -- 172.21.15.3:0/4119615663 learned_addr learned my addr 172.21.15.3:0/4119615663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131dd62640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f13181a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 -- 172.21.15.3:0/4119615663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 msgr2=0x7f13181a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 0x7f13181a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 -- 172.21.15.3:0/4119615663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 msgr2=0x7f13181a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f13181a71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.695+0000 7f131e563640 1 -- 172.21.15.3:0/4119615663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13181ac170 con 0x7f131811b380 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.696+0000 7f131e563640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f130800e6f0 tx=0x7f130800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:07.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.696+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130800cab0 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.696+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1308005500 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.696+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308010430 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.697+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13181ac460 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.697+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181b3d40 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.699+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f131806bc80 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.701+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f13080046c0 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.702+0000 7f130effd640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 0x7f13040a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.703+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f13080cf470 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.703+0000 7f131dd62640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 0x7f13040a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.706+0000 7f131dd62640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 0x7f13040a5210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:06.710+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13080541b0 con 0x7f131811b380 2024-09-12T02:26:07.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.013+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13181a8520 con 0x7f131811b380 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.014+0000 7f130effd640 1 -- 172.21.15.3:0/4119615663 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11951 (secure 0 0 0) 0x7f130809c3e0 con 0x7f131811b380 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":40,"btime":"2024-09-12T02:26:06:997762+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:26:06.372067+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[14619,24482,24508,24487,24493]},"id":1}]} 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 40 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 msgr2=0x7f13040a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:07.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 0x7f13040a5210 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f1314005d20 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 msgr2=0x7f13181aba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f130800e6f0 tx=0x7f130800ebb0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 shutdown_connections 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f13040a2d50 0x7f13040a5210 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f131811b380 0x7f13181aba90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f131806c1c0 0x7f13181a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 --2- 172.21.15.3:0/4119615663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f131806b8a0 0x7f13181a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 >> 172.21.15.3:0/4119615663 conn(0x7f1318071390 msgr2=0x7f1318071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 shutdown_connections 2024-09-12T02:26:07.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:07.016+0000 7f131ed64640 1 -- 172.21.15.3:0/4119615663 wait complete. 2024-09-12T02:26:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:08 smithi026 ceph-mon[35496]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 19 MiB/s wr, 4.37k op/s 2024-09-12T02:26:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:08 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:08.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:08 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4119615663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:08.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:08 smithi003 ceph-mon[31476]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 19 MiB/s wr, 4.37k op/s 2024-09-12T02:26:08.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:08 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:08.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:08 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4119615663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:08 smithi092 ceph-mon[36089]: pgmap v568: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 19 MiB/s wr, 4.37k op/s 2024-09-12T02:26:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:08 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:08.289 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:08 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4119615663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:08.913 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:08.951 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T02:26:08.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T02:26:09.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 -- 172.21.15.3:0/1644471788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 msgr2=0x7f6158104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 --2- 172.21.15.3:0/1644471788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158104e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f614c00a910 tx=0x7f614c02a600 comp rx=0 tx=0).stop 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 -- 172.21.15.3:0/1644471788 shutdown_connections 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 --2- 172.21.15.3:0/1644471788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 --2- 172.21.15.3:0/1644471788 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158104e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 --2- 172.21.15.3:0/1644471788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 -- 172.21.15.3:0/1644471788 >> 172.21.15.3:0/1644471788 conn(0x7f6158077230 msgr2=0x7f61580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 -- 172.21.15.3:0/1644471788 shutdown_connections 2024-09-12T02:26:09.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.480+0000 7f615d087640 1 -- 172.21.15.3:0/1644471788 wait complete. 2024-09-12T02:26:09.482 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:09 smithi003 ceph-mon[31476]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.481+0000 7f615d087640 1 Processor -- start 2024-09-12T02:26:09.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 -- start start 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fd40 con 0x7f6158100430 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fbc0 con 0x7f6158105560 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615d087640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f615810fec0 con 0x7f6158100de0 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:09.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.482+0000 7f615759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615659c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40684/0 (socket says 172.21.15.3:40684) 2024-09-12T02:26:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:50244/0 (socket says 172.21.15.3:50244) 2024-09-12T02:26:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615659c640 1 -- 172.21.15.3:0/2225370934 learned_addr learned my addr 172.21.15.3:0/2225370934 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f6156d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45968/0 (socket says 172.21.15.3:45968) 2024-09-12T02:26:09.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 -- 172.21.15.3:0/2225370934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 msgr2=0x7f6158196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 -- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 msgr2=0x7f6158196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 -- 172.21.15.3:0/2225370934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819afd0 con 0x7f6158105560 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615759e640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6150012be0 tx=0x7f61500095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150009e40 con 0x7f6158105560 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819b2c0 con 0x7f6158105560 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.483+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6150003e50 con 0x7f6158105560 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.484+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615001b760 con 0x7f6158105560 2024-09-12T02:26:09.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.484+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a2ba0 con 0x7f6158105560 2024-09-12T02:26:09.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.489+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61580759d0 con 0x7f6158105560 2024-09-12T02:26:09.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.494+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6150004330 con 0x7f6158105560 2024-09-12T02:26:09.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.494+0000 7f613ffff640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 0x7f61400a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:09.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.495+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f61500d2da0 con 0x7f6158105560 2024-09-12T02:26:09.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.497+0000 7f6156d9d640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 0x7f61400a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:09.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.501+0000 7f6156d9d640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 0x7f61400a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6148009bd0 tx=0x7f6148005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:09.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.504+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f615009fc90 con 0x7f6158105560 2024-09-12T02:26:09.606 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:09.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:09 smithi026 ceph-mon[35496]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:09.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:09 smithi092 ceph-mon[36089]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:26:09.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.891+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f615802d7c0 con 0x7f6158105560 2024-09-12T02:26:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.893+0000 7f613ffff640 1 -- 172.21.15.3:0/2225370934 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7f615009f590 con 0x7f6158105560 2024-09-12T02:26:09.893 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:09.893 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"btime":"2024-09-12T02:26:08:004227+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:26:07.197355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:26:09.893 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 msgr2=0x7f61400a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 0x7f61400a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6148009bd0 tx=0x7f6148005f90 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 msgr2=0x7f615819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6150012be0 tx=0x7f61500095b0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 shutdown_connections 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f61400a2c80 0x7f61400a5140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6158105560 0x7f615819a8f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6158100de0 0x7f6158196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 --2- 172.21.15.3:0/2225370934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6158100430 0x7f6158196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 >> 172.21.15.3:0/2225370934 conn(0x7f6158077230 msgr2=0x7f6158103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 shutdown_connections 2024-09-12T02:26:09.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:09.895+0000 7f615d087640 1 -- 172.21.15.3:0/2225370934 wait complete. 2024-09-12T02:26:10.705 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:10 smithi003 ceph-mon[31476]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 15 MiB/s wr, 3.41k op/s 2024-09-12T02:26:10.705 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:10 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2225370934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:10.737 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24508 quiesce db --cancel --all' 2024-09-12T02:26:10.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24508 quiesce db --cancel --all 2024-09-12T02:26:10.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:10 smithi026 ceph-mon[35496]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 15 MiB/s wr, 3.41k op/s 2024-09-12T02:26:10.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:10 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2225370934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:10.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:10 smithi092 ceph-mon[36089]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 15 MiB/s wr, 3.41k op/s 2024-09-12T02:26:10.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:10 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2225370934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 -- 172.21.15.3:0/398768731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870106fc0 msgr2=0x7f187010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 --2- 172.21.15.3:0/398768731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870106fc0 0x7f187010eaf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1868009ca0 tx=0x7f186802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 -- 172.21.15.3:0/398768731 shutdown_connections 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 --2- 172.21.15.3:0/398768731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870106fc0 0x7f187010eaf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 --2- 172.21.15.3:0/398768731 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100d20 0x7f1870106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 --2- 172.21.15.3:0/398768731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870100370 0x7f1870100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.991+0000 7f1877848640 1 -- 172.21.15.3:0/398768731 >> 172.21.15.3:0/398768731 conn(0x7f1870077210 msgr2=0x7f18700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.993+0000 7f1877848640 1 -- 172.21.15.3:0/398768731 shutdown_connections 2024-09-12T02:26:10.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.993+0000 7f1877848640 1 -- 172.21.15.3:0/398768731 wait complete. 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.994+0000 7f1877848640 1 Processor -- start 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.994+0000 7f1877848640 1 -- start start 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100370 0x7f18701012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870106fc0 0x7f1870101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:10.995 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1870111fa0 con 0x7f1870100370 2024-09-12T02:26:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1870111e20 con 0x7f1870106fc0 2024-09-12T02:26:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1877848640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1870112120 con 0x7f1870100d20 2024-09-12T02:26:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1874de3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1874de3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40702/0 (socket says 172.21.15.3:40702) 2024-09-12T02:26:10.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1874de3640 1 -- 172.21.15.3:0/2007741745 learned_addr learned my addr 172.21.15.3:0/2007741745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f18755e4640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100370 0x7f18701012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.995+0000 7f1875de5640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870106fc0 0x7f1870101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 -- 172.21.15.3:0/2007741745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870106fc0 msgr2=0x7f1870101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870106fc0 0x7f1870101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 -- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100370 msgr2=0x7f18701012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100370 0x7f18701012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 -- 172.21.15.3:0/2007741745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18701aacd0 con 0x7f1870100d20 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f1874de3640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186000c970 tx=0x7f186000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f18667fc640 1 -- 172.21.15.3:0/2007741745 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860014070 con 0x7f1870100d20 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f18667fc640 1 -- 172.21.15.3:0/2007741745 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1860004440 con 0x7f1870100d20 2024-09-12T02:26:10.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.996+0000 7f18667fc640 1 -- 172.21.15.3:0/2007741745 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860014070 con 0x7f1870100d20 2024-09-12T02:26:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.997+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18701aaf00 con 0x7f1870100d20 2024-09-12T02:26:10.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:10.997+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18701ab300 con 0x7f1870100d20 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.001+0000 7f1879c21740 1 Processor -- start 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.001+0000 7f1879c21740 1 -- start start 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951419850 0x555951419c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951424a30 0x555951426e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55595137e160 con 0x555951424a30 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555951325720 con 0x555951419850 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f1879c21740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55595137a120 con 0x55595141a170 2024-09-12T02:26:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40710/0 (socket says 172.21.15.3:40710) 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 -- 172.21.15.3:0/1275922629 learned_addr learned my addr 172.21.15.3:0/1275922629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 -- 172.21.15.3:0/1275922629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951419850 msgr2=0x555951419c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951419850 0x555951419c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 -- 172.21.15.3:0/1275922629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951424a30 msgr2=0x555951426e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951424a30 0x555951426e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.002+0000 7f18755e4640 1 -- 172.21.15.3:0/1275922629 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559510efba0 con 0x55595141a170 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.003+0000 7f18667fc640 1 -- 172.21.15.3:0/2007741745 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1860017070 con 0x7f1870100d20 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.004+0000 7f18755e4640 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f186c00ac20 tx=0x7f186c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c618e61c00f633c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.004+0000 7f18667fc640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 0x7f185c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.005+0000 7f1875de5640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 0x7f185c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.005+0000 7f185b7fe640 1 -- 172.21.15.3:0/1275922629 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c01d070 con 0x55595141a170 2024-09-12T02:26:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.005+0000 7f185b7fe640 1 -- 172.21.15.3:0/1275922629 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c018b20 con 0x55595141a170 2024-09-12T02:26:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 -- 172.21.15.3:0/1275922629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 msgr2=0x5559514244f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1875de5640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 0x7f185c0a5450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1868009a60 tx=0x7f1868005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f186c00ac20 tx=0x7f186c00f790 comp rx=0 tx=0).stop 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f185b7fe640 1 -- 172.21.15.3:0/1275922629 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010c30 con 0x55595141a170 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f18667fc640 1 -- 172.21.15.3:0/2007741745 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f1860004240 con 0x7f1870100d20 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 -- 172.21.15.3:0/1275922629 shutdown_connections 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951424a30 0x555951426e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x5559514244f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 --2- 172.21.15.3:0/1275922629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951419850 0x555951419c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.006+0000 7f1879c21740 1 -- 172.21.15.3:0/1275922629 >> 172.21.15.3:0/1275922629 conn(0x555951415de0 msgr2=0x5559514161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.007+0000 7f1879c21740 1 -- 172.21.15.3:0/1275922629 shutdown_connections 2024-09-12T02:26:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.007+0000 7f1879c21740 1 -- 172.21.15.3:0/1275922629 wait complete. 2024-09-12T02:26:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.007+0000 7f1879c21740 1 -- start start 2024-09-12T02:26:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.008+0000 7f1879c21740 1 Processor -- start 2024-09-12T02:26:11.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.008+0000 7f1879c21740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.008+0000 7f1879c21740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x555951421f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.008+0000 7f1879c21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x5559514224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1879c21740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x555951427650 con 0x555951419850 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45992/0 (socket says 172.21.15.3:45992) 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 -- 172.21.15.3:0/737794855 learned_addr learned my addr 172.21.15.3:0/737794855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x555951306900 con 0x55595141a170 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1874de3640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x555951421f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5559514277d0 con 0x555951424a30 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1875de5640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x5559514224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 -- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 msgr2=0x5559514224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x5559514224a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 -- 172.21.15.3:0/737794855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 msgr2=0x555951421f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x555951421f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f18755e4640 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555951346c50 con 0x555951419850 2024-09-12T02:26:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.009+0000 7f1874de3640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x555951421f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:26:11.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f18755e4640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f186c018a10 tx=0x7f186c018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a391f1dcf6f6b5db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1879c21740 10 client.0 _ll_register_callbacks cb 0x555951304780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c018b20 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1858ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1858ff9640 1 -- 172.21.15.3:0/737794855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c01d050 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1858ff9640 1 -- 172.21.15.3:0/737794855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f186c005450 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1858ff9640 1 -- 172.21.15.3:0/737794855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f186c005650 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1879c21740 10 client.16674 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.010+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=41}) -- 0x555951433210 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1858ff9640 1 -- 172.21.15.3:0/737794855 <== mon.0 v2:172.21.15.3:3300/0 4 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f186c0107d0 con 0x555951419850 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 10 client.16674 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 10 client.16674 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:26:11.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 20 client.16674 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x555951434560 0x555951436950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 4 client.16674 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595138b890 con 0x555951434560 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.011+0000 7f1874de3640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x555951434560 0x555951436950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.012+0000 7f1858ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.012+0000 7f1874de3640 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x555951434560 0x555951436950 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70630f292d661860 server_cookie=a5f6c8bf42127ba3 in_seq=0 out_seq=0 2024-09-12T02:26:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.012+0000 7f1858ff9640 1 -- 172.21.15.3:0/737794855 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55595138b890 con 0x555951434560 2024-09-12T02:26:11.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.012+0000 7f1858ff9640 10 client.16674 handle_command_reply: tid=0 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 client.16674 shutdown 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 msgr2=0x555951421a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f186c018a10 tx=0x7f186c018a40 comp rx=0 tx=0).stop 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 shutdown_connections 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x555951434560 0x555951436950 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1858ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x5559514224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x555951421f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 --2- 172.21.15.3:0/737794855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x555951421a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 >> 172.21.15.3:0/737794855 conn(0x555951415de0 msgr2=0x55595141bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 shutdown_connections 2024-09-12T02:26:11.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 1 -- 172.21.15.3:0/737794855 wait complete. 2024-09-12T02:26:11.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.013+0000 7f1879c21740 20 client.16674 trim_cache size 0 max 16384 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 Processor -- start 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 -- start start 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f18755e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f18755e4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45996/0 (socket says 172.21.15.3:45996) 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x55595143bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f18755e4640 1 -- 172.21.15.3:0/3897301335 learned_addr learned my addr 172.21.15.3:0/3897301335 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x55595143c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b3c20 con 0x555951419850 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b57c0 con 0x55595141a170 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.043+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b42f0 con 0x555951424a30 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f1875de5640 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x55595143c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f1874de3640 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x55595143bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f18755e4640 1 -- 172.21.15.3:0/3897301335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 msgr2=0x55595143c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f18755e4640 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x55595143c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f18755e4640 1 -- 172.21.15.3:0/3897301335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 msgr2=0x55595143bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f18755e4640 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x55595143bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.044+0000 7f18755e4640 1 -- 172.21.15.3:0/3897301335 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595141d600 con 0x555951419850 2024-09-12T02:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.045+0000 7f18755e4640 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f186c018a70 tx=0x7f186c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d15411b8876a667 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.046+0000 7f185affd640 1 -- 172.21.15.3:0/3897301335 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c01d040 con 0x555951419850 2024-09-12T02:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.046+0000 7f185affd640 1 -- 172.21.15.3:0/3897301335 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c018b20 con 0x555951419850 2024-09-12T02:26:11.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.046+0000 7f185affd640 1 -- 172.21.15.3:0/3897301335 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c0054b0 con 0x555951419850 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 msgr2=0x55595141c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f186c018a70 tx=0x7f186c023000 comp rx=0 tx=0).stop 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 shutdown_connections 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555951424a30 0x55595143c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55595141a170 0x55595143bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 --2- 172.21.15.3:0/3897301335 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x555951419850 0x55595141c2b0 secure :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f186c018a70 tx=0x7f186c023000 comp rx=0 tx=0).stop 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.047+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 >> 172.21.15.3:0/3897301335 conn(0x555951415de0 msgr2=0x555951422200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.048+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 shutdown_connections 2024-09-12T02:26:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.048+0000 7f1879c21740 1 -- 172.21.15.3:0/3897301335 wait complete. 2024-09-12T02:26:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.048+0000 7f1879c21740 1 -- start start 2024-09-12T02:26:11.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.049+0000 7f1879c21740 1 Processor -- start 2024-09-12T02:26:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1879c21740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x55595142ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1879c21740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 0x55595142cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1879c21740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 0x555951497e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1874de3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 0x55595142cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1874de3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 0x55595142cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:50290/0 (socket says 172.21.15.3:50290) 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1874de3640 1 -- 172.21.15.3:0/1887564146 learned_addr learned my addr 172.21.15.3:0/1887564146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b50f0 con 0x55595142d4f0 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.050+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b4a20 con 0x555951424a30 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f18701b1c00 con 0x55595141a170 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1879c21740 10 client.0 _ll_register_callbacks cb 0x7f186c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:11.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1875de5640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 0x555951497e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1875de5640 1 -- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 msgr2=0x55595142ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1875de5640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x55595142ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1875de5640 1 -- 172.21.15.3:0/1887564146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 msgr2=0x55595142cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.051+0000 7f1875de5640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 0x55595142cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.052+0000 7f1875de5640 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555951498420 con 0x55595142d4f0 2024-09-12T02:26:11.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.052+0000 7f1875de5640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 0x555951497e30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1868006010 tx=0x7f186802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f38b6702bc1c6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.052+0000 7f1858ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:26:11.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.052+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868004450 con 0x55595142d4f0 2024-09-12T02:26:11.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.053+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f18680045f0 con 0x55595142d4f0 2024-09-12T02:26:11.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.053+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c018b20 con 0x55595142d4f0 2024-09-12T02:26:11.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.053+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f18680047f0 con 0x55595142d4f0 2024-09-12T02:26:11.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.053+0000 7f1879c21740 10 client.16680 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:11.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.053+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5559514b07f0 con 0x55595142d4f0 2024-09-12T02:26:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.054+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c018b20 con 0x55595142d4f0 2024-09-12T02:26:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.054+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f1868043040 con 0x55595142d4f0 2024-09-12T02:26:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 10 client.16680 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 10 client.16680 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:26:11.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 20 client.16680 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5559514b0fd0 0x5559514b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 4 client.16680 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T02:26:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.055+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x555951385be0 con 0x5559514b0fd0 2024-09-12T02:26:11.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.056+0000 7f18755e4640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5559514b0fd0 0x5559514b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.060+0000 7f18755e4640 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5559514b0fd0 0x5559514b3450 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784a6ed9dc3f5ca6 server_cookie=824876b13db13358 in_seq=0 out_seq=0 2024-09-12T02:26:11.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.060+0000 7f1858ff9640 10 client.16680 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.076+0000 7f1858ff9640 1 -- 172.21.15.3:0/1887564146 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555951385be0 con 0x5559514b0fd0 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.076+0000 7f1858ff9640 10 client.16680 handle_command_reply: tid=0 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.076+0000 7f1879c21740 1 client.16680 shutdown 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 msgr2=0x555951497e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 0x555951497e30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1868006010 tx=0x7f186802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 shutdown_connections 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x5559514b0fd0 0x5559514b3450 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55595142d4f0 0x555951497e30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555951424a30 0x55595142cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 --2- 172.21.15.3:0/1887564146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55595141a170 0x55595142ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 >> 172.21.15.3:0/1887564146 conn(0x555951415de0 msgr2=0x5559514146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1858ff9640 0 client.16680 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 shutdown_connections 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 1 -- 172.21.15.3:0/1887564146 wait complete. 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.077+0000 7f1879c21740 20 client.16680 trim_cache size 0 max 16384 2024-09-12T02:26:11.080 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 41, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 18, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "714bcbeb": { 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 18, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 90.0, 2024-09-12T02:26:11.081 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 140.2, 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:11.082 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.079+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 msgr2=0x7f185c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.079+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 0x7f185c0a5450 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1868009a60 tx=0x7f1868005f70 comp rx=0 tx=0).stop 2024-09-12T02:26:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.079+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 msgr2=0x7f1870101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.079+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186000c970 tx=0x7f186000ce30 comp rx=0 tx=0).stop 2024-09-12T02:26:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 shutdown_connections 2024-09-12T02:26:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f185c0a2f90 0x7f185c0a5450 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1870106fc0 0x7f1870101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1870100d20 0x7f1870101830 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 --2- 172.21.15.3:0/2007741745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1870100370 0x7f18701012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 >> 172.21.15.3:0/2007741745 conn(0x7f1870077210 msgr2=0x7f1870105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 shutdown_connections 2024-09-12T02:26:11.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:11.080+0000 7f1877848640 1 -- 172.21.15.3:0/2007741745 wait complete. 2024-09-12T02:26:11.332 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:11.757 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '714bcbeb', quiesce took 7.9 seconds. Will release after: 12.249999999999998 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.782+0000 7f9817e77640 1 -- 172.21.15.3:0/1786122871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98101025e0 msgr2=0x7f9810108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.782+0000 7f9817e77640 1 --2- 172.21.15.3:0/1786122871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98101025e0 0x7f9810108e70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f980c009900 tx=0x7f980c02a400 comp rx=0 tx=0).stop 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.782+0000 7f9817e77640 1 -- 172.21.15.3:0/1786122871 shutdown_connections 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.782+0000 7f9817e77640 1 --2- 172.21.15.3:0/1786122871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98101025e0 0x7f9810108e70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.782+0000 7f9817e77640 1 --2- 172.21.15.3:0/1786122871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9810101c20 0x7f98101020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.783+0000 7f9817e77640 1 --2- 172.21.15.3:0/1786122871 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.783+0000 7f9817e77640 1 -- 172.21.15.3:0/1786122871 >> 172.21.15.3:0/1786122871 conn(0x7f98100fc190 msgr2=0x7f98100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.783+0000 7f9817e77640 1 -- 172.21.15.3:0/1786122871 shutdown_connections 2024-09-12T02:26:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.784+0000 7f9817e77640 1 -- 172.21.15.3:0/1786122871 wait complete. 2024-09-12T02:26:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.784+0000 7f9817e77640 1 Processor -- start 2024-09-12T02:26:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.784+0000 7f9817e77640 1 -- start start 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9810101c20 0x7f9810198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98101025e0 0x7f981019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f981010d7b0 con 0x7f9810100a30 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f981010d630 con 0x7f98101025e0 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.785+0000 7f9817e77640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f981010d930 con 0x7f9810101c20 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.786+0000 7f9815c13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.786+0000 7f9815c13640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46022/0 (socket says 172.21.15.3:46022) 2024-09-12T02:26:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.786+0000 7f9815c13640 1 -- 172.21.15.3:0/236683249 learned_addr learned my addr 172.21.15.3:0/236683249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.786+0000 7f9815c13640 1 -- 172.21.15.3:0/236683249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9810101c20 msgr2=0x7f9810198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.786+0000 7f9815c13640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9810101c20 0x7f9810198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.787+0000 7f9815c13640 1 -- 172.21.15.3:0/236683249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98101025e0 msgr2=0x7f981019f3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.787+0000 7f9815c13640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98101025e0 0x7f981019f3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.787+0000 7f9815c13640 1 -- 172.21.15.3:0/236683249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981019fa60 con 0x7f9810100a30 2024-09-12T02:26:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.787+0000 7f9815c13640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f980000d770 tx=0x7f980000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.789+0000 7f9806ffd640 1 -- 172.21.15.3:0/236683249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800017070 con 0x7f9810100a30 2024-09-12T02:26:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.789+0000 7f9806ffd640 1 -- 172.21.15.3:0/236683249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9800009c30 con 0x7f9810100a30 2024-09-12T02:26:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.790+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981019fd50 con 0x7f9810100a30 2024-09-12T02:26:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.790+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98101a0240 con 0x7f9810100a30 2024-09-12T02:26:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.790+0000 7f9806ffd640 1 -- 172.21.15.3:0/236683249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800003dd0 con 0x7f9810100a30 2024-09-12T02:26:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.794+0000 7f981ac0c740 1 Processor -- start 2024-09-12T02:26:11.795 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.796+0000 7f9806ffd640 1 -- 172.21.15.3:0/236683249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9800003f70 con 0x7f9810100a30 2024-09-12T02:26:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.797+0000 7f9806ffd640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 0x7f97fc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.798+0000 7f9816414640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 0x7f97fc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 -- start start 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ecc56432f0 con 0x55ecc56e0260 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ecc55eb270 con 0x55ecc56df920 2024-09-12T02:26:11.804 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f981ac0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ecc563f2b0 con 0x55ecc56e3330 2024-09-12T02:26:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.804+0000 7f9816414640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 0x7f97fc0a5190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f980c02a920 tx=0x7f980c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40768/0 (socket says 172.21.15.3:40768) 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 -- 172.21.15.3:0/2864896551 learned_addr learned my addr 172.21.15.3:0/2864896551 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 -- 172.21.15.3:0/2864896551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 msgr2=0x55ecc56dfd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56dfd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 -- 172.21.15.3:0/2864896551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 msgr2=0x55ecc56e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 -- 172.21.15.3:0/2864896551 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc55ea340 con 0x55ecc56e3330 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.805+0000 7f9816414640 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f980c033000 tx=0x7f980c03a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5452da9ec5d7833 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.806+0000 7f97fbfff640 1 -- 172.21.15.3:0/2864896551 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c035040 con 0x55ecc56e3330 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.806+0000 7f97fbfff640 1 -- 172.21.15.3:0/2864896551 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c04c070 con 0x55ecc56e3330 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.806+0000 7f97fbfff640 1 -- 172.21.15.3:0/2864896551 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c047b80 con 0x55ecc56e3330 2024-09-12T02:26:11.807 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.806+0000 7f9806ffd640 1 -- 172.21.15.3:0/236683249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f98000ce970 con 0x7f9810100a30 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 -- 172.21.15.3:0/2864896551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 msgr2=0x55ecc56f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f980c033000 tx=0x7f980c03a000 comp rx=0 tx=0).stop 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 -- 172.21.15.3:0/2864896551 shutdown_connections 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e3330 0x55ecc56f02e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2864896551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.807+0000 7f981ac0c740 1 -- 172.21.15.3:0/2864896551 >> 172.21.15.3:0/2864896551 conn(0x55ecc56db010 msgr2=0x55ecc56db410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.808 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.808+0000 7f981ac0c740 1 -- 172.21.15.3:0/2864896551 shutdown_connections 2024-09-12T02:26:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.808+0000 7f981ac0c740 1 -- 172.21.15.3:0/2864896551 wait complete. 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.808+0000 7f981ac0c740 1 -- start start 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.808+0000 7f981ac0c740 1 Processor -- start 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 0x55ecc56f9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ecc56f0b10 con 0x55ecc56e0260 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55ecc55cc290 con 0x55ecc56e3330 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ecc56f0c90 con 0x55ecc56df920 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f981ac0c740 10 client.0 _ll_register_callbacks cb 0x55ecc55fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9815c13640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9815412640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.810 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9815c13640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40776/0 (socket says 172.21.15.3:40776) 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9815412640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46036/0 (socket says 172.21.15.3:46036) 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9815c13640 1 -- 172.21.15.3:0/2067132979 learned_addr learned my addr 172.21.15.3:0/2067132979 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.809+0000 7f9816414640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 0x55ecc56f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 -- 172.21.15.3:0/2067132979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 msgr2=0x55ecc56f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 0x55ecc56f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 -- 172.21.15.3:0/2067132979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 msgr2=0x55ecc56f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 -- 172.21.15.3:0/2067132979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc5356210 con 0x55ecc56df920 2024-09-12T02:26:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9816414640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 0x55ecc56f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:26:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f9815c13640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f980009b1d0 tx=0x7f980009ba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=663a29bc502cf572 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f97f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:26:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000b4060 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f98000978b0 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.810+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f980c04c070 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98000bb050 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 10 client.26230 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ecc5422620 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f980c04c070 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f980009f170 con 0x55ecc56df920 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 10 client.26230 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 10 client.26230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 20 client.26230 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc56fe020 0x55ecc5700410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 4 client.26230 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:26:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.811+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc5650a20 con 0x55ecc56fe020 2024-09-12T02:26:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.812+0000 7f9815412640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc56fe020 0x55ecc5700410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.813+0000 7f9815412640 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc56fe020 0x55ecc5700410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854e31750beb7f1a server_cookie=8ca6b18bf50c5249 in_seq=0 out_seq=0 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.813+0000 7f97f97fa640 10 client.26230 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f97f97fa640 1 -- 172.21.15.3:0/2067132979 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecc5650a20 con 0x55ecc56fe020 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f97f97fa640 10 client.26230 handle_command_reply: tid=0 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 client.26230 shutdown 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 msgr2=0x55ecc56f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f980009b1d0 tx=0x7f980009ba00 comp rx=0 tx=0).stop 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 shutdown_connections 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc56fe020 0x55ecc5700410 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.815 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e3330 0x55ecc56f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e0260 0x55ecc56f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 --2- 172.21.15.3:0/2067132979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56f23f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 >> 172.21.15.3:0/2067132979 conn(0x55ecc56db010 msgr2=0x55ecc56dc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f97f97fa640 0 client.26230 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 shutdown_connections 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.814+0000 7f981ac0c740 1 -- 172.21.15.3:0/2067132979 wait complete. 2024-09-12T02:26:11.816 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.815+0000 7f981ac0c740 20 client.26230 trim_cache size 0 max 16384 2024-09-12T02:26:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.840+0000 7f981ac0c740 1 Processor -- start 2024-09-12T02:26:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.841+0000 7f981ac0c740 1 -- start start 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.841+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.841+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.841+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.842+0000 7f981ac0c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9810072610 con 0x55ecc56e3330 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.842+0000 7f981ac0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f98100741b0 con 0x55ecc56e0260 2024-09-12T02:26:11.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.842+0000 7f981ac0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9810072ce0 con 0x55ecc56df920 2024-09-12T02:26:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.844+0000 7f9815412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.844+0000 7f9815412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:50310/0 (socket says 172.21.15.3:50310) 2024-09-12T02:26:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.844+0000 7f9815412640 1 -- 172.21.15.3:0/608154535 learned_addr learned my addr 172.21.15.3:0/608154535 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.844+0000 7f9815c13640 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.845+0000 7f9815412640 1 -- 172.21.15.3:0/608154535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 msgr2=0x55ecc56dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.845+0000 7f9815412640 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56dbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.845+0000 7f9815412640 1 -- 172.21.15.3:0/608154535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 msgr2=0x55ecc56dc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.845+0000 7f9815412640 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.845 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.845+0000 7f9815412640 1 -- 172.21.15.3:0/608154535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc56dae30 con 0x55ecc56e0260 2024-09-12T02:26:11.846 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.846+0000 7f9815412640 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9810199350 tx=0x7f9808007680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3a8177172b0bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.849 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.849+0000 7f97fb7fe640 1 -- 172.21.15.3:0/608154535 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808011e60 con 0x55ecc56e0260 2024-09-12T02:26:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.850+0000 7f97fb7fe640 1 -- 172.21.15.3:0/608154535 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808019b90 con 0x55ecc56e0260 2024-09-12T02:26:11.850 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.850+0000 7f97fb7fe640 1 -- 172.21.15.3:0/608154535 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800e3b0 con 0x55ecc56e0260 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 -- 172.21.15.3:0/608154535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 msgr2=0x55ecc56f9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9810199350 tx=0x7f9808007680 comp rx=0 tx=0).stop 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 -- 172.21.15.3:0/608154535 shutdown_connections 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56dc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56e0260 0x55ecc56f9930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 --2- 172.21.15.3:0/608154535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56df920 0x55ecc56dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.851+0000 7f981ac0c740 1 -- 172.21.15.3:0/608154535 >> 172.21.15.3:0/608154535 conn(0x55ecc56db010 msgr2=0x55ecc56f21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.852+0000 7f981ac0c740 1 -- 172.21.15.3:0/608154535 shutdown_connections 2024-09-12T02:26:11.852 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.852+0000 7f981ac0c740 1 -- 172.21.15.3:0/608154535 wait complete. 2024-09-12T02:26:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.852+0000 7f981ac0c740 1 -- start start 2024-09-12T02:26:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 Processor -- start 2024-09-12T02:26:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9810073ae0 con 0x55ecc56e3330 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f9810073410 con 0x55ecc56df920 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f98100705f0 con 0x55ecc56e0260 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.853+0000 7f981ac0c740 10 client.0 _ll_register_callbacks cb 0x7f98000035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.854+0000 7f9815412640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.854+0000 7f9815412640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40800/0 (socket says 172.21.15.3:40800) 2024-09-12T02:26:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.854+0000 7f9815412640 1 -- 172.21.15.3:0/3928635962 learned_addr learned my addr 172.21.15.3:0/3928635962 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.855+0000 7f9815412640 1 -- 172.21.15.3:0/3928635962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 msgr2=0x55ecc56f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.855+0000 7f9815412640 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.855+0000 7f9815412640 1 -- 172.21.15.3:0/3928635962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 msgr2=0x55ecc56f83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.855+0000 7f9815412640 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.855+0000 7f9815412640 1 -- 172.21.15.3:0/3928635962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc56f89e0 con 0x55ecc56e0260 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f9815412640 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f980800fed0 tx=0x7f980800ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632a5f1758e9ef9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f97f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980800f830 con 0x55ecc56e0260 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808019b90 con 0x55ecc56e0260 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f9808003970 con 0x55ecc56e0260 2024-09-12T02:26:11.857 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9808003b70 con 0x55ecc56e0260 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f981ac0c740 10 client.26233 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.856+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ecc5768cf0 con 0x55ecc56e0260 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9808019b90 con 0x55ecc56e0260 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f9808022c20 con 0x55ecc56e0260 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f981ac0c740 10 client.26233 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f981ac0c740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f981ac0c740 20 client.26233 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.857+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc57694c0 0x55ecc576b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.858+0000 7f981ac0c740 4 client.26233 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T02:26:11.858 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.858+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecc564ad70 con 0x55ecc57694c0 2024-09-12T02:26:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.862+0000 7f9815c13640 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc57694c0 0x55ecc576b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.863+0000 7f97f97fa640 10 client.26233 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.863+0000 7f9815c13640 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc57694c0 0x55ecc576b8e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc530fc7711b051 server_cookie=a074460b617691f4 in_seq=0 out_seq=0 2024-09-12T02:26:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.884+0000 7f97f97fa640 1 -- 172.21.15.3:0/3928635962 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecc564ad70 con 0x55ecc57694c0 2024-09-12T02:26:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.884+0000 7f97f97fa640 10 client.26233 handle_command_reply: tid=0 2024-09-12T02:26:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.885+0000 7f981ac0c740 1 client.26233 shutdown 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 msgr2=0x55ecc56f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f980800fed0 tx=0x7f980800ff00 comp rx=0 tx=0).stop 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 shutdown_connections 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55ecc57694c0 0x55ecc576b8e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ecc56e3330 0x55ecc56f83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ecc56e0260 0x55ecc56f73c0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f980800fed0 tx=0x7f980800ff00 comp rx=0 tx=0).stop 2024-09-12T02:26:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 --2- 172.21.15.3:0/3928635962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ecc56df920 0x55ecc56f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.886+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 >> 172.21.15.3:0/3928635962 conn(0x55ecc56db010 msgr2=0x55ecc56d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.888+0000 7f97f97fa640 0 client.26233 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.889+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 shutdown_connections 2024-09-12T02:26:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.889+0000 7f981ac0c740 1 -- 172.21.15.3:0/3928635962 wait complete. 2024-09-12T02:26:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.889+0000 7f981ac0c740 20 client.26233 trim_cache size 0 max 16384 2024-09-12T02:26:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-09-12T02:26:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-09-12T02:26:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-09-12T02:26:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-09-12T02:26:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.892+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 msgr2=0x7f97fc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.892+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 0x7f97fc0a5190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f980c02a920 tx=0x7f980c005fd0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.892+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 msgr2=0x7f9810197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.892+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f980000d770 tx=0x7f980000dc30 comp rx=0 tx=0).stop 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 shutdown_connections 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f97fc0a2cd0 0x7f97fc0a5190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98101025e0 0x7f981019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9810101c20 0x7f9810198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 --2- 172.21.15.3:0/236683249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9810100a30 0x7f9810197e40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:11.893 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.893+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 >> 172.21.15.3:0/236683249 conn(0x7f98100fc190 msgr2=0x7f9810104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.894+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 shutdown_connections 2024-09-12T02:26:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-09-12T02:26:11.894+0000 7f9817e77640 1 -- 172.21.15.3:0/236683249 wait complete. 2024-09-12T02:26:12.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.345+0000 7f2239766640 1 -- 172.21.15.3:0/2039740247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102ef0 msgr2=0x7f223410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.345+0000 7f2239766640 1 --2- 172.21.15.3:0/2039740247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102ef0 0x7f223410da40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ca0 tx=0x7f222c02a950 comp rx=0 tx=0).stop 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 -- 172.21.15.3:0/2039740247 shutdown_connections 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 --2- 172.21.15.3:0/2039740247 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102ef0 0x7f223410da40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 --2- 172.21.15.3:0/2039740247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234102570 0x7f22341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 --2- 172.21.15.3:0/2039740247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234100e20 0x7f2234101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 -- 172.21.15.3:0/2039740247 >> 172.21.15.3:0/2039740247 conn(0x7f22340fc7d0 msgr2=0x7f22340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 -- 172.21.15.3:0/2039740247 shutdown_connections 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 -- 172.21.15.3:0/2039740247 wait complete. 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 Processor -- start 2024-09-12T02:26:12.347 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.346+0000 7f2239766640 1 -- start start 2024-09-12T02:26:12.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 0x7f223419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 0x7f223419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2234110fd0 con 0x7f2234102570 2024-09-12T02:26:12.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2234110e50 con 0x7f2234102ef0 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2239766640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2234111150 con 0x7f2234100e20 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46086/0 (socket says 172.21.15.3:46086) 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 -- 172.21.15.3:0/4284186336 learned_addr learned my addr 172.21.15.3:0/4284186336 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22337fe640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 0x7f223419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2232ffd640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 0x7f223419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 -- 172.21.15.3:0/4284186336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 msgr2=0x7f223419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 0x7f223419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 -- 172.21.15.3:0/4284186336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 msgr2=0x7f223419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 0x7f223419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22327fc640 1 -- 172.21.15.3:0/4284186336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223419f5c0 con 0x7f2234102570 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f22337fe640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 0x7f223419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:26:12.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.347+0000 7f2232ffd640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 0x7f223419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f22327fc640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f222bfff640 1 -- 172.21.15.3:0/4284186336 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224017030 con 0x7f2234102570 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f222bfff640 1 -- 172.21.15.3:0/4284186336 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2224004020 con 0x7f2234102570 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223419f8b0 con 0x7f2234102570 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f222bfff640 1 -- 172.21.15.3:0/4284186336 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240047a0 con 0x7f2234102570 2024-09-12T02:26:12.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.348+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a7190 con 0x7f2234102570 2024-09-12T02:26:12.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.350+0000 7f222bfff640 1 -- 172.21.15.3:0/4284186336 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f22240041d0 con 0x7f2234102570 2024-09-12T02:26:12.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.350+0000 7f223bb3f740 1 Processor -- start 2024-09-12T02:26:12.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.350+0000 7f223bb3f740 1 -- start start 2024-09-12T02:26:12.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7aeb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 0x55f6b7aedfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f6b7a45200 con 0x55f6b7aebbe0 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6b79ebdc0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f223bb3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f6b7a411c0 con 0x55f6b7ae1320 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7aeb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f22337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7aeb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40828/0 (socket says 172.21.15.3:40828) 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f2232ffd640 1 -- 172.21.15.3:0/20019866 learned_addr learned my addr 172.21.15.3:0/20019866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.351+0000 7f222bfff640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 0x7f22200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 0x55f6b7aedfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22337fe640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 0x7f22200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 -- 172.21.15.3:0/20019866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 msgr2=0x55f6b7aeb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7aeb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 -- 172.21.15.3:0/20019866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 msgr2=0x55f6b7ae0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f22327fc640 1 -- 172.21.15.3:0/20019866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b77b6620 con 0x55f6b7aebbe0 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.352+0000 7f222bfff640 1 -- 172.21.15.3:0/4284186336 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f22240cea40 con 0x7f2234102570 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.353+0000 7f22337fe640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 0x7f22200a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f222c006010 tx=0x7f222c005f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.353+0000 7f22327fc640 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 0x55f6b7aedfd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2224097c60 tx=0x7f2224000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef004ed63eabc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.353+0000 7f22297fa640 1 -- 172.21.15.3:0/20019866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222409b290 con 0x55f6b7aebbe0 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.353+0000 7f22297fa640 1 -- 172.21.15.3:0/20019866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222409f640 con 0x55f6b7aebbe0 2024-09-12T02:26:12.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.353+0000 7f22297fa640 1 -- 172.21.15.3:0/20019866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240bb0d0 con 0x55f6b7aebbe0 2024-09-12T02:26:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.354+0000 7f223bb3f740 1 -- 172.21.15.3:0/20019866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 msgr2=0x55f6b7aedfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.354+0000 7f223bb3f740 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 0x55f6b7aedfd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2224097c60 tx=0x7f2224000bc0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 -- 172.21.15.3:0/20019866 shutdown_connections 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7aebbe0 0x55f6b7aedfd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7aeb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 --2- 172.21.15.3:0/20019866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 -- 172.21.15.3:0/20019866 >> 172.21.15.3:0/20019866 conn(0x55f6b7adcf90 msgr2=0x55f6b7add370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 -- 172.21.15.3:0/20019866 shutdown_connections 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.355+0000 7f223bb3f740 1 -- 172.21.15.3:0/20019866 wait complete. 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 -- start start 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 Processor -- start 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae1320 0x55f6b7ae87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7ae8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55f6b7aee7b0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x55f6b79dfc50 con 0x55f6b7ae1320 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f6b7aee930 con 0x55f6b7aebbe0 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f223bb3f740 10 client.0 _ll_register_callbacks cb 0x55f6b79f1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46104/0 (socket says 172.21.15.3:46104) 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.356+0000 7f2232ffd640 1 -- 172.21.15.3:0/3791595974 learned_addr learned my addr 172.21.15.3:0/3791595974 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f22337fe640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7ae8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f22327fc640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae1320 0x55f6b7ae87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 -- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 msgr2=0x55f6b7ae8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7ae8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 -- 172.21.15.3:0/3791595974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae1320 msgr2=0x55f6b7ae87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae1320 0x55f6b7ae87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 -- 172.21.15.3:0/3791595974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b7ae9570 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f2232ffd640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f221c013790 tx=0x7f221c013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70647609e5ad2ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.357+0000 7f221a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c004440 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222409f640 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f221c0119c0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f221c011bc0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f223bb3f740 10 client.16701 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.358+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f6b7b02c20 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222409f640 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f221c025050 con 0x55f6b7ae0a00 2024-09-12T02:26:12.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 10 client.16701 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 10 client.16701 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 20 client.16701 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b04030 0x55f6b7b06420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 4 client.16701 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.359+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b7a52770 con 0x55f6b7b04030 2024-09-12T02:26:12.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.360+0000 7f22327fc640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b04030 0x55f6b7b06420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.360+0000 7f22327fc640 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b04030 0x55f6b7b06420 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b07578031e67a6f server_cookie=20ced98a551d24d7 in_seq=0 out_seq=0 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.360+0000 7f221a7fc640 10 client.16701 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.361+0000 7f221a7fc640 1 -- 172.21.15.3:0/3791595974 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6b7a52770 con 0x55f6b7b04030 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.361+0000 7f221a7fc640 10 client.16701 handle_command_reply: tid=0 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.361+0000 7f223bb3f740 1 client.16701 shutdown 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.361+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 msgr2=0x55f6b7aeb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.361+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f221c013790 tx=0x7f221c013c50 comp rx=0 tx=0).stop 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 shutdown_connections 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b04030 0x55f6b7b06420 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7ae8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae1320 0x55f6b7ae87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 --2- 172.21.15.3:0/3791595974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7aeb530 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f221a7fc640 0 client.16701 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 >> 172.21.15.3:0/3791595974 conn(0x55f6b7adcf90 msgr2=0x55f6b7adc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 shutdown_connections 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 1 -- 172.21.15.3:0/3791595974 wait complete. 2024-09-12T02:26:12.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.362+0000 7f223bb3f740 20 client.16701 trim_cache size 0 max 16384 2024-09-12T02:26:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.387+0000 7f223bb3f740 1 Processor -- start 2024-09-12T02:26:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.390+0000 7f223bb3f740 1 -- start start 2024-09-12T02:26:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.390+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7aebbe0 0x55f6b7b0bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f223bb3f740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22341ab490 con 0x55f6b7ae0a00 2024-09-12T02:26:12.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f223bb3f740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22341ad030 con 0x55f6b7aebbe0 2024-09-12T02:26:12.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f223bb3f740 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22341abb60 con 0x55f6b7ae1320 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f22327fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f22327fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40846/0 (socket says 172.21.15.3:40846) 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f22327fc640 1 -- 172.21.15.3:0/2697244366 learned_addr learned my addr 172.21.15.3:0/2697244366 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.391+0000 7f2232ffd640 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.392+0000 7f22327fc640 1 -- 172.21.15.3:0/2697244366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7aebbe0 msgr2=0x55f6b7b0bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.392+0000 7f22327fc640 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7aebbe0 0x55f6b7b0bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.392+0000 7f22327fc640 1 -- 172.21.15.3:0/2697244366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 msgr2=0x55f6b7ae3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.392+0000 7f22327fc640 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.392+0000 7f22327fc640 1 -- 172.21.15.3:0/2697244366 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b7b06aa0 con 0x55f6b7ae1320 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.393+0000 7f22327fc640 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22240cc8a0 tx=0x7f222409b190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf74842097767b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.393+0000 7f2228ff9640 1 -- 172.21.15.3:0/2697244366 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222409f640 con 0x55f6b7ae1320 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.393+0000 7f2228ff9640 1 -- 172.21.15.3:0/2697244366 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2224099330 con 0x55f6b7ae1320 2024-09-12T02:26:12.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.393+0000 7f2228ff9640 1 -- 172.21.15.3:0/2697244366 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222409f1b0 con 0x55f6b7ae1320 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.394+0000 7f223bb3f740 1 -- 172.21.15.3:0/2697244366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 msgr2=0x55f6b7b002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.394+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22240cc8a0 tx=0x7f222409b190 comp rx=0 tx=0).stop 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 -- 172.21.15.3:0/2697244366 shutdown_connections 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7aebbe0 0x55f6b7b0bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7ae1320 0x55f6b7b002a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2697244366 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae0a00 0x55f6b7ae3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 -- 172.21.15.3:0/2697244366 >> 172.21.15.3:0/2697244366 conn(0x55f6b7adcf90 msgr2=0x55f6b7aea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 -- 172.21.15.3:0/2697244366 shutdown_connections 2024-09-12T02:26:12.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 -- 172.21.15.3:0/2697244366 wait complete. 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.395+0000 7f223bb3f740 1 -- start start 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 Processor -- start 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae1320 0x55f6b7af8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f2232ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.3:50360/0 (socket says 172.21.15.3:50360) 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f2232ffd640 1 -- 172.21.15.3:0/2025833349 learned_addr learned my addr 172.21.15.3:0/2025833349 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7af8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f22341ac960 con 0x55f6b7ae1320 2024-09-12T02:26:12.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f22341ac290 con 0x55f6b7ae0a00 2024-09-12T02:26:12.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f22341a94d0 con 0x55f6b7aebbe0 2024-09-12T02:26:12.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.396+0000 7f223bb3f740 10 client.0 _ll_register_callbacks cb 0x7f221c009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:26:12.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f22327fc640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae1320 0x55f6b7af8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f22337fe640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7af8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f2232ffd640 1 -- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 msgr2=0x55f6b7af8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f2232ffd640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7af8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f2232ffd640 1 -- 172.21.15.3:0/2025833349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae1320 msgr2=0x55f6b7af8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f2232ffd640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae1320 0x55f6b7af8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.397+0000 7f2232ffd640 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b7b6dfb0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.398+0000 7f2232ffd640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221c013790 tx=0x7f221c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df889c2c856ba1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.398+0000 7f221a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.398+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c011d60 con 0x55f6b7ae0a00 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.398+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f221c01c920 con 0x55f6b7ae0a00 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.399+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6b7a4cc80 con 0x55f6b7ae0a00 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.399+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f221c01cb20 con 0x55f6b7ae0a00 2024-09-12T02:26:12.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.400+0000 7f223bb3f740 10 client.26087 fetch_fsmap learned FSMap version 41 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.400+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f6b7b6f2d0 con 0x55f6b7ae0a00 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f6b7a4cc80 con 0x55f6b7ae0a00 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) ==== 8382+0+0 (secure 0 0 0) 0x7f221c025040 con 0x55f6b7ae0a00 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 10 client.26087 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 10 client.26087 resolve_mds: validated gid 24508 aka daemon mds.i 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 20 client.26087 populate_metadata read hostname 'smithi003' 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b6f9a0 0x55f6b7b71dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 4 client.26087 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T02:26:12.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.401+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 --> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f6b79c91e0 con 0x55f6b7b6f9a0 2024-09-12T02:26:12.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.404+0000 7f22327fc640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b6f9a0 0x55f6b7b71dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:12.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.411+0000 7f221a7fc640 10 client.26087 ms_handle_connect on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:12.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.411+0000 7f22327fc640 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b6f9a0 0x55f6b7b71dc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8acac992e3c6d767 server_cookie=44a70201ce4ea5f3 in_seq=0 out_seq=0 2024-09-12T02:26:12.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.412+0000 7f221a7fc640 1 -- 172.21.15.3:0/2025833349 <== mds.0 v2:172.21.15.92:6838/1571545450 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f6b79c91e0 con 0x55f6b7b6f9a0 2024-09-12T02:26:12.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.412+0000 7f221a7fc640 10 client.26087 handle_command_reply: tid=0 2024-09-12T02:26:12.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.412+0000 7f223bb3f740 1 client.26087 shutdown 2024-09-12T02:26:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 msgr2=0x55f6b7af84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221c013790 tx=0x7f221c0117b0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 shutdown_connections 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:6838/1571545450,v1:172.21.15.92:6839/1571545450] conn(0x55f6b7b6f9a0 0x55f6b7b71dc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f6b7aebbe0 0x55f6b7af8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f221a7fc640 0 client.26087 ms_handle_reset on v2:172.21.15.92:6838/1571545450 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55f6b7ae1320 0x55f6b7af8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 --2- 172.21.15.3:0/2025833349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6b7ae0a00 0x55f6b7af84c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.415+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 >> 172.21.15.3:0/2025833349 conn(0x55f6b7adcf90 msgr2=0x55f6b7ae2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.416+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 shutdown_connections 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.416+0000 7f223bb3f740 1 -- 172.21.15.3:0/2025833349 wait complete. 2024-09-12T02:26:12.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.416+0000 7f223bb3f740 20 client.26087 trim_cache size 0 max 16384 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 41, 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24508, 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 19, 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": {} 2024-09-12T02:26:12.417 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.418+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 msgr2=0x7f22200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.418+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 0x7f22200a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f222c006010 tx=0x7f222c005f10 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.418+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 msgr2=0x7f223419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.418+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400de10 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 shutdown_connections 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f22200a2b80 0x7f22200a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2234102ef0 0x7f223419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2234102570 0x7f223419ab50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 --2- 172.21.15.3:0/4284186336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2234100e20 0x7f223419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.419+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 >> 172.21.15.3:0/4284186336 conn(0x7f22340fc7d0 msgr2=0x7f2234069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.420+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 shutdown_connections 2024-09-12T02:26:12.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:12.420+0000 7f2239766640 1 -- 172.21.15.3:0/4284186336 wait complete. 2024-09-12T02:26:12.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:12 smithi026 ceph-mon[35496]: pgmap v570: 97 pgs: 97 active+clean; 7.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2024-09-12T02:26:12.772 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:12 smithi003 ceph-mon[31476]: pgmap v570: 97 pgs: 97 active+clean; 7.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2024-09-12T02:26:12.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:12 smithi092 ceph-mon[36089]: pgmap v570: 97 pgs: 97 active+clean; 7.7 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 13 MiB/s wr, 3.13k op/s 2024-09-12T02:26:13.217 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T02:26:13.218 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T02:26:13.234 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:26:13.270 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T02:26:13.280 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T02:26:13.280 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-09-12T02:26:13.281 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T02:26:13.292 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 -- 172.21.15.3:0/730613495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 msgr2=0x7fee08073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 --2- 172.21.15.3:0/730613495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee08073170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009900 tx=0x7fedf002a410 comp rx=0 tx=0).stop 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 -- 172.21.15.3:0/730613495 shutdown_connections 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 --2- 172.21.15.3:0/730613495 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee08119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 --2- 172.21.15.3:0/730613495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee08073170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 --2- 172.21.15.3:0/730613495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 0x7fee08109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 -- 172.21.15.3:0/730613495 >> 172.21.15.3:0/730613495 conn(0x7fee0806ecd0 msgr2=0x7fee08071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 -- 172.21.15.3:0/730613495 shutdown_connections 2024-09-12T02:26:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.477+0000 7fee0deae640 1 -- 172.21.15.3:0/730613495 wait complete. 2024-09-12T02:26:13.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.478+0000 7fee0deae640 1 Processor -- start 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.478+0000 7fee0deae640 1 -- start start 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee081a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee081a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 0x7fee081ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee0811c5e0 con 0x7fee080736b0 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fee0811c460 con 0x7fee08072d10 2024-09-12T02:26:13.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fedfffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee081a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fedfffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee081a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46148/0 (socket says 172.21.15.3:46148) 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fedfffff640 1 -- 172.21.15.3:0/1671394374 learned_addr learned my addr 172.21.15.3:0/1671394374 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee077fe640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee081a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee07fff640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 0x7fee081ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.479+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee0811c760 con 0x7fee08109700 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 -- 172.21.15.3:0/1671394374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 msgr2=0x7fee081ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 0x7fee081ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 -- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 msgr2=0x7fee081a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee081a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee081ae830 con 0x7fee08072d10 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee077fe640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee081a6c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedf800e9f0 tx=0x7fedf800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf800ccc0 con 0x7fee08072d10 2024-09-12T02:26:13.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee081aeb20 con 0x7fee08072d10 2024-09-12T02:26:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081af0c0 con 0x7fee08072d10 2024-09-12T02:26:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf8010910 con 0x7fee08072d10 2024-09-12T02:26:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.480+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8018960 con 0x7fee08072d10 2024-09-12T02:26:13.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.481+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede80052c0 con 0x7fee08072d10 2024-09-12T02:26:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.483+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fedf8010ac0 con 0x7fee08072d10 2024-09-12T02:26:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.483+0000 7fee057fa640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 0x7fedf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.484+0000 7fedfffff640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 0x7fedf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.484+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fedf80cf3d0 con 0x7fee08072d10 2024-09-12T02:26:13.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.484+0000 7fedfffff640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 0x7fedf40a50d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fedf00096c0 tx=0x7fedf00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:13.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.491+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedf809bc70 con 0x7fee08072d10 2024-09-12T02:26:13.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.775+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fede80055b0 con 0x7fee08072d10 2024-09-12T02:26:13.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.777+0000 7fee057fa640 1 -- 172.21.15.3:0/1671394374 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7fedf809ff70 con 0x7fee08072d10 2024-09-12T02:26:13.778 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:13.778 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:26:13.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 msgr2=0x7fedf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 0x7fedf40a50d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fedf00096c0 tx=0x7fedf00315b0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 msgr2=0x7fee081a6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee081a6c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedf800e9f0 tx=0x7fedf800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 shutdown_connections 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fedf40a2c10 0x7fedf40a50d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee08109700 0x7fee081ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee080736b0 0x7fee081a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 --2- 172.21.15.3:0/1671394374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee08072d10 0x7fee081a6c20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.781+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 >> 172.21.15.3:0/1671394374 conn(0x7fee0806ecd0 msgr2=0x7fee08071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.782+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 shutdown_connections 2024-09-12T02:26:13.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.782+0000 7fee0deae640 1 -- 172.21.15.3:0/1671394374 wait complete. 2024-09-12T02:26:13.795 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:11:52.858011+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T02:16:25.722169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T02:16:28.093977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:26:13.795 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.939+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/1932049101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e4073440 msgr2=0x7fe9e4073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.939+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/1932049101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e4073440 0x7fe9e4073840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009900 tx=0x7fe9d802a440 comp rx=0 tx=0).stop 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/1932049101 shutdown_connections 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/1932049101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e410adc0 0x7fe9e4111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/1932049101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e410a400 0x7fe9e410a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/1932049101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e4073440 0x7fe9e4073840 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/1932049101 >> 172.21.15.3:0/1932049101 conn(0x7fe9e406ec70 msgr2=0x7fe9e40710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:13.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.940+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/1932049101 shutdown_connections 2024-09-12T02:26:13.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.941+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/1932049101 wait complete. 2024-09-12T02:26:13.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.942+0000 7fe9e9ad0640 1 Processor -- start 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 -- start start 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4073440 0x7fe9e4109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e410a400 0x7fe9e41a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4115000 con 0x7fe9e410adc0 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4114e80 con 0x7fe9e4073440 2024-09-12T02:26:13.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e9ad0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4115180 con 0x7fe9e410a400 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46160/0 (socket says 172.21.15.3:46160) 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.943+0000 7fe9e3fff640 1 -- 172.21.15.3:0/2557466901 learned_addr learned my addr 172.21.15.3:0/2557466901 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 -- 172.21.15.3:0/2557466901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e410a400 msgr2=0x7fe9e41a5570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e410a400 0x7fe9e41a5570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 -- 172.21.15.3:0/2557466901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4073440 msgr2=0x7fe9e4109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:13.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4073440 0x7fe9e4109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:13.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 -- 172.21.15.3:0/2557466901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e41a84a0 con 0x7fe9e410adc0 2024-09-12T02:26:13.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.944+0000 7fe9e3fff640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b990 tx=0x7fe9dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:13.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.945+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc01d070 con 0x7fe9e410adc0 2024-09-12T02:26:13.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.945+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e419f3a0 con 0x7fe9e410adc0 2024-09-12T02:26:13.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.945+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e419f970 con 0x7fe9e410adc0 2024-09-12T02:26:13.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.945+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9dc018920 con 0x7fe9e410adc0 2024-09-12T02:26:13.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.946+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc007af0 con 0x7fe9e410adc0 2024-09-12T02:26:13.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.946+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d00052c0 con 0x7fe9e410adc0 2024-09-12T02:26:13.949 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.950+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe9dc020070 con 0x7fe9e410adc0 2024-09-12T02:26:13.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.951+0000 7fe9e0ff9640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 0x7fe9cc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:13.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.951+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fe9dc0d2f40 con 0x7fe9e410adc0 2024-09-12T02:26:13.951 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.951+0000 7fe9e37fe640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 0x7fe9cc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:13.952 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.952+0000 7fe9e37fe640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 0x7fe9cc0a5250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009850 tx=0x7fe9d8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:13.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:13.957+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9dc09feb0 con 0x7fe9e410adc0 2024-09-12T02:26:14.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.273+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9d0005880 con 0x7fe9e410adc0 2024-09-12T02:26:14.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.275+0000 7fe9e0ff9640 1 -- 172.21.15.3:0/2557466901 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fe9dc09f7b0 con 0x7fe9e410adc0 2024-09-12T02:26:14.275 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-09-12T02:26:14.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 msgr2=0x7fe9cc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 0x7fe9cc0a5250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009850 tx=0x7fe9d8031400 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 msgr2=0x7fe9e41a7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00b990 tx=0x7fe9dc00be50 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 shutdown_connections 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fe9cc0a2d90 0x7fe9cc0a5250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9e410adc0 0x7fe9e41a7b90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9e410a400 0x7fe9e41a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 --2- 172.21.15.3:0/2557466901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9e4073440 0x7fe9e4109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 >> 172.21.15.3:0/2557466901 conn(0x7fe9e406ec70 msgr2=0x7fe9e410d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.278+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 shutdown_connections 2024-09-12T02:26:14.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.279+0000 7fe9e9ad0640 1 -- 172.21.15.3:0/2557466901 wait complete. 2024-09-12T02:26:14.294 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:26:14.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 -- 172.21.15.3:0/122109591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60881092f0 msgr2=0x7f6088109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.459 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 --2- 172.21.15.3:0/122109591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60881092f0 0x7f6088109750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6084009cd0 tx=0x7f608402a8f0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 -- 172.21.15.3:0/122109591 shutdown_connections 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 --2- 172.21.15.3:0/122109591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6088109c90 0x7f6088119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 --2- 172.21.15.3:0/122109591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60881092f0 0x7f6088109750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 --2- 172.21.15.3:0/122109591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 0x7f60880737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.458+0000 7f608f4d0640 1 -- 172.21.15.3:0/122109591 >> 172.21.15.3:0/122109591 conn(0x7f608806ec70 msgr2=0x7f60880710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.459+0000 7f608f4d0640 1 -- 172.21.15.3:0/122109591 shutdown_connections 2024-09-12T02:26:14.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.459+0000 7f608f4d0640 1 -- 172.21.15.3:0/122109591 wait complete. 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.459+0000 7f608f4d0640 1 Processor -- start 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 -- start start 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 0x7f60881a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 0x7f60881a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 0x7f60881ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f608811c5c0 con 0x7f60880733f0 2024-09-12T02:26:14.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f608811c440 con 0x7f60881092f0 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608f4d0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f608811c740 con 0x7f6088109c90 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608da6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 0x7f60881ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608da6d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 0x7f60881ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.3:40908/0 (socket says 172.21.15.3:40908) 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608da6d640 1 -- 172.21.15.3:0/2780144160 learned_addr learned my addr 172.21.15.3:0/2780144160 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608ca6b640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 0x7f60881a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.460+0000 7f608d26c640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 0x7f60881a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 -- 172.21.15.3:0/2780144160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 msgr2=0x7f60881ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 0x7f60881ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 -- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 msgr2=0x7f60881a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 0x7f60881a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 -- 172.21.15.3:0/2780144160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881ae8b0 con 0x7f60881092f0 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.461+0000 7f608ca6b640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 0x7f60881a71b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6084009bf0 tx=0x7f6084030da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.462+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084003d80 con 0x7f60881092f0 2024-09-12T02:26:14.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.462+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6084040070 con 0x7f60881092f0 2024-09-12T02:26:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.463+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840047d0 con 0x7f60881092f0 2024-09-12T02:26:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.463+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60881aeb40 con 0x7f60881092f0 2024-09-12T02:26:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.463+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881af010 con 0x7f60881092f0 2024-09-12T02:26:14.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.464+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608810ec60 con 0x7f60881092f0 2024-09-12T02:26:14.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.466+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6084031880 con 0x7f60881092f0 2024-09-12T02:26:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.467+0000 7f607e7fc640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 0x7f60740a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.467+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f60840f06e0 con 0x7f60881092f0 2024-09-12T02:26:14.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.467+0000 7f608d26c640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 0x7f60740a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.468+0000 7f608d26c640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 0x7f60740a5250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60780058c0 tx=0x7f607800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:14.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.474+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f608404a060 con 0x7f60881092f0 2024-09-12T02:26:14.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:14 smithi026 ceph-mon[35496]: pgmap v571: 97 pgs: 97 active+clean; 6.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.3 MiB/s wr, 685 op/s 2024-09-12T02:26:14.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:14 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1671394374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:14.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:14 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2557466901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:26:14.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:14 smithi003 ceph-mon[31476]: pgmap v571: 97 pgs: 97 active+clean; 6.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.3 MiB/s wr, 685 op/s 2024-09-12T02:26:14.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:14 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1671394374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:14.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:14 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2557466901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:26:14.782 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.782+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f608802da10 con 0x7f60881092f0 2024-09-12T02:26:14.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.783+0000 7f607e7fc640 1 -- 172.21.15.3:0/2780144160 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f60840bd650 con 0x7f60881092f0 2024-09-12T02:26:14.783 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-09-12T02:26:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 msgr2=0x7f60740a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.785 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 0x7f60740a5250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60780058c0 tx=0x7f607800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 msgr2=0x7f60881a71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 0x7f60881a71b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6084009bf0 tx=0x7f6084030da0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 shutdown_connections 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f60740a2d90 0x7f60740a5250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6088109c90 0x7f60881ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60881092f0 0x7f60881a71b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 --2- 172.21.15.3:0/2780144160 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f60880733f0 0x7f60881a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 >> 172.21.15.3:0/2780144160 conn(0x7f608806ec70 msgr2=0x7f60880717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 shutdown_connections 2024-09-12T02:26:14.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.785+0000 7f608f4d0640 1 -- 172.21.15.3:0/2780144160 wait complete. 2024-09-12T02:26:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:14 smithi092 ceph-mon[36089]: pgmap v571: 97 pgs: 97 active+clean; 6.6 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.3 MiB/s wr, 685 op/s 2024-09-12T02:26:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:14 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1671394374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:14.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:14 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2557466901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:26:14.796 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 -- 172.21.15.3:0/4032798593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 msgr2=0x7fad70101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 --2- 172.21.15.3:0/4032798593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad70101ef0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad5800b9f0 tx=0x7fad5802c0a0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 -- 172.21.15.3:0/4032798593 shutdown_connections 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 --2- 172.21.15.3:0/4032798593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 0x7fad70108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 --2- 172.21.15.3:0/4032798593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad70101ef0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 --2- 172.21.15.3:0/4032798593 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 -- 172.21.15.3:0/4032798593 >> 172.21.15.3:0/4032798593 conn(0x7fad700fbfe0 msgr2=0x7fad700fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 -- 172.21.15.3:0/4032798593 shutdown_connections 2024-09-12T02:26:14.939 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.938+0000 7fad75a01640 1 -- 172.21.15.3:0/4032798593 wait complete. 2024-09-12T02:26:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.939+0000 7fad75a01640 1 Processor -- start 2024-09-12T02:26:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.939+0000 7fad75a01640 1 -- start start 2024-09-12T02:26:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad7019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.941 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 0x7fad701a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010b440 con 0x7fad70100880 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010b2c0 con 0x7fad70101a70 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad75a01640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010b5c0 con 0x7fad70102430 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad6effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46192/0 (socket says 172.21.15.3:46192) 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad7019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad6effd640 1 -- 172.21.15.3:0/2028568266 learned_addr learned my addr 172.21.15.3:0/2028568266 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad6f7fe640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 0x7fad701a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 -- 172.21.15.3:0/2028568266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 msgr2=0x7fad701a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 0x7fad701a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 -- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 msgr2=0x7fad70199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:14.942 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.940+0000 7fad667fc640 1 -- 172.21.15.3:0/2028568266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad701a1af0 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad667fc640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad7019a410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad5802c5c0 tx=0x7fad580042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58039070 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701a1d80 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad58009a20 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad58034b50 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.941+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad701a21b0 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.942+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad70107ce0 con 0x7fad70101a70 2024-09-12T02:26:14.943 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.943+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fad58044060 con 0x7fad70101a70 2024-09-12T02:26:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.944+0000 7fad749ff640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 0x7fad5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.944+0000 7fad6effd640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 0x7fad5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.944+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fad580ee570 con 0x7fad70101a70 2024-09-12T02:26:14.944 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.944+0000 7fad6effd640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 0x7fad5c0a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fad60009c60 tx=0x7fad60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:14.948 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:14.948+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad580bb460 con 0x7fad70101a70 2024-09-12T02:26:15.248 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-09-12T02:26:15.250 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.247+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad7002da30 con 0x7fad70101a70 2024-09-12T02:26:15.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.247+0000 7fad749ff640 1 -- 172.21.15.3:0/2028568266 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fad580bad60 con 0x7fad70101a70 2024-09-12T02:26:15.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 msgr2=0x7fad5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:15.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 0x7fad5c0a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fad60009c60 tx=0x7fad60009260 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 msgr2=0x7fad7019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad7019a410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad5802c5c0 tx=0x7fad580042f0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 shutdown_connections 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fad5c0a2b90 0x7fad5c0a5050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad70102430 0x7fad701a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70101a70 0x7fad7019a410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 --2- 172.21.15.3:0/2028568266 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fad70100880 0x7fad70199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 >> 172.21.15.3:0/2028568266 conn(0x7fad700fbfe0 msgr2=0x7fad70104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 shutdown_connections 2024-09-12T02:26:15.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.253+0000 7fad75a01640 1 -- 172.21.15.3:0/2028568266 wait complete. 2024-09-12T02:26:15.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 -- 172.21.15.3:0/2102191178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10000737b0 msgr2=0x7f1000073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 --2- 172.21.15.3:0/2102191178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10000737b0 0x7f1000073bb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009860 tx=0x7f0fe802a340 comp rx=0 tx=0).stop 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 -- 172.21.15.3:0/2102191178 shutdown_connections 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 --2- 172.21.15.3:0/2102191178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f100010adc0 0x7f1000111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 --2- 172.21.15.3:0/2102191178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f100010a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 --2- 172.21.15.3:0/2102191178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10000737b0 0x7f1000073bb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.419+0000 7f1004b3b640 1 -- 172.21.15.3:0/2102191178 >> 172.21.15.3:0/2102191178 conn(0x7f100006ef10 msgr2=0x7f1000071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:15.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.420+0000 7f1004b3b640 1 -- 172.21.15.3:0/2102191178 shutdown_connections 2024-09-12T02:26:15.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.420+0000 7f1004b3b640 1 -- 172.21.15.3:0/2102191178 wait complete. 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.420+0000 7f1004b3b640 1 Processor -- start 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 -- start start 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10000737b0 0x7f10001a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f10001a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113de0 con 0x7f100010adc0 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113c60 con 0x7f100010a400 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.421+0000 7f1004b3b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1000113f60 con 0x7f10000737b0 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.422+0000 7f0ffed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.422+0000 7f0ffed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46210/0 (socket says 172.21.15.3:46210) 2024-09-12T02:26:15.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.422+0000 7f0ffed9d640 1 -- 172.21.15.3:0/169670957 learned_addr learned my addr 172.21.15.3:0/169670957 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffdd9b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f10001a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffed9d640 1 -- 172.21.15.3:0/169670957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10000737b0 msgr2=0x7f10001a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffed9d640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10000737b0 0x7f10001a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffed9d640 1 -- 172.21.15.3:0/169670957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 msgr2=0x7f10001a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffed9d640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f10001a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffed9d640 1 -- 172.21.15.3:0/169670957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10001ae930 con 0x7f100010adc0 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.423+0000 7f0ffdd9b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f10001a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:26:15.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.425+0000 7f0ffed9d640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800eda0 tx=0x7f0ff800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.426+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8003b70 con 0x7f100010adc0 2024-09-12T02:26:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.426+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff8003d10 con 0x7f100010adc0 2024-09-12T02:26:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.426+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8017910 con 0x7f100010adc0 2024-09-12T02:26:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.426+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10001aec20 con 0x7f100010adc0 2024-09-12T02:26:15.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.426+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10001af120 con 0x7f100010adc0 2024-09-12T02:26:15.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.428+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100010cd30 con 0x7f100010adc0 2024-09-12T02:26:15.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.430+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0ff8004020 con 0x7f100010adc0 2024-09-12T02:26:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.430+0000 7f0ff77fe640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 0x7f0fec0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.431+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f0ff80cf4a0 con 0x7f100010adc0 2024-09-12T02:26:15.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.432+0000 7f0ffe59c640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 0x7f0fec0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:15.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.432+0000 7f0ffe59c640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 0x7f0fec0a5250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005b60 tx=0x7f0fe8005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:15.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.437+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ff809c390 con 0x7f100010adc0 2024-09-12T02:26:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:15 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2780144160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:15 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2028568266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:15 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2780144160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:15 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2028568266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:15 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2780144160' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:15 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2028568266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:26:15.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.819+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1000062ef0 con 0x7f100010adc0 2024-09-12T02:26:15.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.834+0000 7f0ff77fe640 1 -- 172.21.15.3:0/169670957 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740362 (secure 0 0 0) 0x7f0ff809bc90 con 0x7f100010adc0 2024-09-12T02:26:15.844 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:15.844 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 20, 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 14150, 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2546806763 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.845 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2546806763 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/2546806763", 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-09-12T02:09:36.346976+0000", 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.846 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24109, 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:15.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.849 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-12T02:26:15.862 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.863 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:15.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:15.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:26:15.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:26:15.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.869 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:15.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.872 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.874 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-12T02:26:15.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:15.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-12T02:26:15.878 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.881 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:26:15.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.881 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.881 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-12T02:26:15.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-12T02:26:15.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.888 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:26:15.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-12T02:26:15.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.896 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.899 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:26:15.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-12T02:26:15.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.908 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:26:15.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-12T02:26:15.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:26:15.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-12T02:26:15.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.924 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.925 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.927 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.930 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.931 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-12T02:26:15.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:15.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:26:15.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-12T02:26:15.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.951 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.954 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:15.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:15.958 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:15.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-12T02:26:15.962 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:26:15.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:26:15.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.971 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.972 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.974 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.978 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.985 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:26:15.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:26:15.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:26:15.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.992 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.995 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.997 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.997 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:15.998 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:15.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.000 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:26:16.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:26:16.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.010 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.016 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.018 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-12T02:26:16.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.021 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.022 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.024 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.030 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.035 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.036 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.039 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.043 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-12T02:26:16.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.045 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.046 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-12T02:26:16.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-12T02:26:16.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.061 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-12T02:26:16.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-12T02:26:16.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-12T02:26:16.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.069 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.073 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.075 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.076 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-12T02:26:16.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-12T02:26:16.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.089 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.091 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.093 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.096 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:26:16.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.104 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.105 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.113 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.115 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.118 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.119 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:16.121 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.124 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.126 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.134 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.139 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.140 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.141 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.145 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.149 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.155 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:26:16.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-12T02:26:16.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-12T02:26:16.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.163 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-12T02:26:16.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:16.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.178 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-12T02:26:16.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:16.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.190 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.191 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.192 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.196 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.197 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.203 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.209 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.210 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-12T02:26:16.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-12T02:26:16.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.224 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.227 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-12T02:26:16.228 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.230 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.234 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.236 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.238 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.240 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.243 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.245 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.246 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.248 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.251 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:26:16.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.271 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.279 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.283 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.284 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-12T02:26:16.293 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.296 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.297 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.300 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-12T02:26:16.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-12T02:26:16.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.305 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24110, 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-12T02:26:16.306 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.308 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:26:16.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-12T02:26:16.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-12T02:26:16.316 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-12T02:26:16.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:26:16.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:26:16.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-12T02:26:16.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.321 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.323 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:26:16.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-12T02:26:16.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-12T02:26:16.326 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-12T02:26:16.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-12T02:26:16.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.329 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.330 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:16.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:26:16.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:26:16.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:26:16.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.341 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:26:16.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-12T02:26:16.348 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.359 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-12T02:26:16.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:26:16.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:26:16.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-12T02:26:16.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.382 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.383 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-12T02:26:16.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:26:16.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-12T02:26:16.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-12T02:26:16.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.397 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.398 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.400 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:26:16.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:26:16.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:26:16.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.406 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.407 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:26:16.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.415 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.416 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.434 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.443 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.447 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:26:16.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.449 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.467 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-12T02:26:16.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:26:16.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.475 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.476 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.481 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.484 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-12T02:26:16.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-12T02:26:16.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-12T02:26:16.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.494 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.497 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.498 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-12T02:26:16.501 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.503 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.504 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.512 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.515 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.516 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-12T02:26:16.520 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.521 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.526 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.529 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.531 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.537 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-12T02:26:16.539 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:26:16.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.541 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.543 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.544 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.549 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.551 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.552 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.557 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.558 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.562 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.564 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.565 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.566 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.567 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.568 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.570 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.572 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.575 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.579 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.581 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.582 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.583 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.586 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.588 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.590 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.593 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.594 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.596 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.597 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.598 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-12T02:26:16.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.605 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:26:16.606 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.607 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.608 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.621 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.622 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-12T02:26:16.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:16.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.636 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.637 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.638 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.640 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.645 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.647 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.648 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.653 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.658 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.662 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.667 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-12T02:26:16.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-12T02:26:16.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.674 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.676 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.677 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.679 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.684 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.685 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.687 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.689 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.690 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.693 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.694 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.695 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.698 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.703 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.707 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.709 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.711 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.713 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-12T02:26:16.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.715 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.716 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.719 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.720 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.722 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.723 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.726 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-12T02:26:16.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.729 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.730 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.738 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.742 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.743 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "restful" 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.744 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.745 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.746 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.749 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.750 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.751 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.752 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-09-12T02:26:16.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-09-12T02:26:16.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-09-12T02:26:16.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-09-12T02:26:16.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.759 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:16.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.762 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-09-12T02:26:16.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-09-12T02:26:16.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.770 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.774 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T02:26:16.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T02:26:16.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.782 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-09-12T02:26:16.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.788 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.789 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-09-12T02:26:16.792 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-09-12T02:26:16.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-09-12T02:26:16.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-09-12T02:26:16.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-09-12T02:26:16.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.797 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-09-12T02:26:16.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-09-12T02:26:16.799 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-09-12T02:26:16.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-09-12T02:26:16.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-09-12T02:26:16.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-09-12T02:26:16.803 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-09-12T02:26:16.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-09-12T02:26:16.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-09-12T02:26:16.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.809 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-09-12T02:26:16.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-09-12T02:26:16.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-09-12T02:26:16.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-09-12T02:26:16.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-09-12T02:26:16.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-09-12T02:26:16.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.816 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T02:26:16.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T02:26:16.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-09-12T02:26:16.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-09-12T02:26:16.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-09-12T02:26:16.826 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-09-12T02:26:16.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T02:26:16.830 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T02:26:16.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T02:26:16.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T02:26:16.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T02:26:16.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T02:26:16.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-09-12T02:26:16.836 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T02:26:16.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T02:26:16.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T02:26:16.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T02:26:16.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T02:26:16.841 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.842 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.844 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.847 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.850 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.854 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T02:26:16.861 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T02:26:16.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.863 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T02:26:16.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.875 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.877 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.878 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.882 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.889 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.890 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.891 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T02:26:16.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.893 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.894 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.895 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.902 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.904 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.910 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.913 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:16.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.917 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.918 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.921 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.922 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.929 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.930 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.933 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-09-12T02:26:16.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:16.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.938 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T02:26:16.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T02:26:16.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.941 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-09-12T02:26:16.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.954 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.955 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.960 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.963 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.965 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.967 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.968 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.969 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.970 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.971 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-09-12T02:26:16.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.975 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.978 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.985 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.987 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.988 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.991 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.995 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:16.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:16.997 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:16.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:16.999 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-09-12T02:26:17.009 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.011 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.015 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.017 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.019 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.020 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.022 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.029 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.030 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.031 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.032 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.038 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-09-12T02:26:17.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T02:26:17.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.042 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.045 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:17.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:17.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-09-12T02:26:17.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.055 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.059 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-09-12T02:26:17.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T02:26:17.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.067 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.075 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.077 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.079 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.081 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.083 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.086 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.087 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.093 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-09-12T02:26:17.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.101 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.102 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.106 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.108 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.115 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.119 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.121 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.125 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.127 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T02:26:17.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-09-12T02:26:17.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.135 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-09-12T02:26:17.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-09-12T02:26:17.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-09-12T02:26:17.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.143 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.147 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.150 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.158 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.159 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.160 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.162 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-12T02:26:17.164 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.165 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-09-12T02:26:17.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-09-12T02:26:17.174 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.178 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-09-12T02:26:17.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-09-12T02:26:17.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-09-12T02:26:17.186 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-09-12T02:26:17.187 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-12T02:26:17.188 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-12T02:26:17.189 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-09-12T02:26:17.190 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 3, 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T02:26:17.191 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3089375163 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 798360927 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.192 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2150548678 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-09-12T02:26:17.193 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3496966164 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-09-12T02:26:17.194 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:17.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:16 smithi092 ceph-mon[36089]: pgmap v572: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 929 KiB/s wr, 444 op/s 2024-09-12T02:26:17.195 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:16 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/169670957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:26:17.196 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:16 smithi026 ceph-mon[35496]: pgmap v572: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 929 KiB/s wr, 444 op/s 2024-09-12T02:26:17.196 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:16 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/169670957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:26:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 msgr2=0x7f0fec0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 0x7f0fec0a5250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8005b60 tx=0x7f0fe8005a90 comp rx=0 tx=0).stop 2024-09-12T02:26:17.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 msgr2=0x7f10001ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800eda0 tx=0x7f0ff800c6e0 comp rx=0 tx=0).stop 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 shutdown_connections 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f0fec0a2d90 0x7f0fec0a5250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f100010adc0 0x7f10001ae290 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f100010a400 0x7f10001a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 --2- 172.21.15.3:0/169670957 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10000737b0 0x7f10001a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.842+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 >> 172.21.15.3:0/169670957 conn(0x7f100006ef10 msgr2=0x7f10001169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.843+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 shutdown_connections 2024-09-12T02:26:17.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:15.843+0000 7f1004b3b640 1 -- 172.21.15.3:0/169670957 wait complete. 2024-09-12T02:26:17.251 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T02:26:17.251 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T02:26:17.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:16 smithi003 ceph-mon[31476]: pgmap v572: 97 pgs: 97 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 929 KiB/s wr, 444 op/s 2024-09-12T02:26:17.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:16 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/169670957' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T02:26:17.521 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:18.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:18 smithi026 ceph-mon[35496]: pgmap v573: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 63 KiB/s wr, 250 op/s 2024-09-12T02:26:18.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:18 smithi003 ceph-mon[31476]: pgmap v573: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 63 KiB/s wr, 250 op/s 2024-09-12T02:26:18.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:18 smithi092 ceph-mon[36089]: pgmap v573: 97 pgs: 97 active+clean; 1.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 63 KiB/s wr, 250 op/s 2024-09-12T02:26:20.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:20 smithi026 ceph-mon[35496]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 KiB/s wr, 228 op/s 2024-09-12T02:26:20.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:20 smithi003 ceph-mon[31476]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 KiB/s wr, 228 op/s 2024-09-12T02:26:20.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:20 smithi092 ceph-mon[36089]: pgmap v574: 97 pgs: 97 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 KiB/s wr, 228 op/s 2024-09-12T02:26:22.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:22 smithi026 ceph-mon[35496]: pgmap v575: 97 pgs: 97 active+clean; 149 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 242 op/s 2024-09-12T02:26:22.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:22 smithi003 ceph-mon[31476]: pgmap v575: 97 pgs: 97 active+clean; 149 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 242 op/s 2024-09-12T02:26:22.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:22 smithi092 ceph-mon[36089]: pgmap v575: 97 pgs: 97 active+clean; 149 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 242 op/s 2024-09-12T02:26:23.100 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1053, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 794, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 362957, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 355739, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 11, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 6, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.091077471, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.030724994, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.065962224, 2024-09-12T02:26:23.101 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.006806362, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 793, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.064641222, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000081514 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1706, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000488752, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000286 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:23.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 51, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 178, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 12974, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 444145, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 18, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 5, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.030219151, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.015347501, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.013027930, 2024-09-12T02:26:23.103 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000242529, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 176, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.036706002, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000208556 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 61, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000009596, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000157 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-12T02:26:23.104 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 299680 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 308, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 276, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 924424, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 186442, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 8, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 5, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.034143542, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.013822804, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.027398319, 2024-09-12T02:26:23.105 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000633221, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 275, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.046835658, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000170311 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 105, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000016320, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000155 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 866960, 2024-09-12T02:26:23.106 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 107488 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 97, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.005987813, 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000061730 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 17, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000413892, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000024346 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 6, 2024-09-12T02:26:23.108 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 6, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 6, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.067777414, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.011296235 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 11, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2024-09-12T02:26:23.109 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 17, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 1, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 16, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 13, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 20, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 4, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 31, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 20, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2024-09-12T02:26:23.110 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 5, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 2, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 2, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 3, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 2, 2024-09-12T02:26:23.111 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 175, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 214153, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 2, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 2, 2024-09-12T02:26:23.112 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 6, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 4, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 11, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 1, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 2, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 4 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2024-09-12T02:26:23.113 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 3, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 2, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 8, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 2, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 2, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:23.114 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_path": 5, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_inode": 57, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 2, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.115 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 27, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 27, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 2, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 2, 2024-09-12T02:26:23.116 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 4241587, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 0, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 19, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 1.650091778, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.086846935 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 0 2024-09-12T02:26:23.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 22, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 34, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 12, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 21, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 32, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 11, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 17, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 42, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 25, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 8, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 8, 2024-09-12T02:26:23.118 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 55484, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 223520 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 12, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 3, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 6, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 10, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 10, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.119 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.120 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.121 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.122 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2024-09-12T02:26:23.124 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 4, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.001051205, 2024-09-12T02:26:23.125 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000262801 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.126 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.127 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.128 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.129 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.066726209, 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.033363104 2024-09-12T02:26:23.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 1, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 2, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 1, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 1, 2024-09-12T02:26:23.131 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 4, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 4, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 1555, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:23.132 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:23.133 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:23.134 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 270487, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 91, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:23.135 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 157, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 88621, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 346, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2024-09-12T02:26:23.136 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 75, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 59713, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 32, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 75, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 75, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 2.027973859, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.027039651 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 75, 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 101 2024-09-12T02:26:23.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 75, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 3, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 72, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 10, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 1, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 20, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 11, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2024-09-12T02:26:23.138 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 14, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 2, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 20, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:23.139 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2024-09-12T02:26:23.140 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 10, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 10, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 0, 2024-09-12T02:26:23.141 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 20, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 0, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 3 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 1, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "osd-slow-ops": { 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 56 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.142 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 2, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 2, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed_ops": 2, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 2, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:23.143 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 1408, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1159302, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 1408, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1408, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1159302, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 75, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 59915, 2024-09-12T02:26:23.145 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 75, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 59915, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:23.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 75, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 75, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 75, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 75, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 2, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 202, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 2, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 202, 2024-09-12T02:26:23.148 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x557025b80500": { 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 27, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 47283, 2024-09-12T02:26:23.149 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 27, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 19, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 47283, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:23.150 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:24.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:24 smithi026 ceph-mon[35496]: pgmap v576: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 178 op/s 2024-09-12T02:26:24.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:24 smithi003 ceph-mon[31476]: pgmap v576: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 178 op/s 2024-09-12T02:26:24.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:24 smithi092 ceph-mon[36089]: pgmap v576: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 178 op/s 2024-09-12T02:26:26.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:26 smithi026 ceph-mon[35496]: pgmap v577: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 158 op/s 2024-09-12T02:26:26.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:26 smithi003 ceph-mon[31476]: pgmap v577: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 158 op/s 2024-09-12T02:26:26.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:26 smithi092 ceph-mon[36089]: pgmap v577: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.7 KiB/s wr, 158 op/s 2024-09-12T02:26:28.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:28 smithi026 ceph-mon[35496]: pgmap v578: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 75 op/s 2024-09-12T02:26:28.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:28 smithi003 ceph-mon[31476]: pgmap v578: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 75 op/s 2024-09-12T02:26:28.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:28 smithi092 ceph-mon[36089]: pgmap v578: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 75 op/s 2024-09-12T02:26:30.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:30 smithi026 ceph-mon[35496]: pgmap v579: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 49 op/s 2024-09-12T02:26:30.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:30 smithi003 ceph-mon[31476]: pgmap v579: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 49 op/s 2024-09-12T02:26:30.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:30 smithi092 ceph-mon[36089]: pgmap v579: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 49 op/s 2024-09-12T02:26:32.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:32 smithi026 ceph-mon[35496]: pgmap v580: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 43 op/s 2024-09-12T02:26:32.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:32 smithi003 ceph-mon[31476]: pgmap v580: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 43 op/s 2024-09-12T02:26:32.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:32 smithi092 ceph-mon[36089]: pgmap v580: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 43 op/s 2024-09-12T02:26:33.040 INFO:tasks.check_counter:Daemon mds.a mds.exported=2 2024-09-12T02:26:33.040 INFO:tasks.check_counter:Daemon mds.a mds.imported=2 2024-09-12T02:26:33.040 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-12T02:26:33.040 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T02:26:33.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T02:26:33.305 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:34.680 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:34 smithi003 ceph-mon[31476]: pgmap v581: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T02:26:34.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:34 smithi026 ceph-mon[35496]: pgmap v581: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T02:26:34.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:34 smithi092 ceph-mon[36089]: pgmap v581: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 178, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 173, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 741872, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 105808, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.028623635, 2024-09-12T02:26:34.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.011465217, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.022075605, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 172, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.015181248, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000088263 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:34.791 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 741872, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 105808 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 5, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 133, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3184, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 102032, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 3, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:34.792 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.012687567, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.007911104, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.003272825, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 132, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.013781559, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000104405 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:34.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 102032 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.000237087, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T02:26:34.794 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000164326, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:34.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:34.796 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:34.797 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:34.798 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148984, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-09-12T02:26:34.799 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:34.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 179, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 716756, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 179, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 179, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 716756, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:34.801 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:34.802 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:35.571 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T02:26:35.571 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T02:26:35.571 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T02:26:35.571 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T02:26:35.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T02:26:35.840 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 538, 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 540, 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 180713, 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 171085, 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 6, 2024-09-12T02:26:36.381 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.065356887, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.024676017, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.014851128, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.018411681, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 538, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.033226788, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000061759 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.382 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1198, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 1177, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1156009, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 422887, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:36.383 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.107714836, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.029640828, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.048798908, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.031236307, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1177, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.055303729, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000046987 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.384 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 853824, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 104512 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 131, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1508, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 303605, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:36.385 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.010483128, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.007415867, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001943148, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000050160, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 129, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.005801381, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000044971 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:36.386 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.387 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.387 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.387 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:36.387 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 302640 2024-09-12T02:26:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-12T02:26:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 1, 2024-09-12T02:26:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-12T02:26:36.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.389 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-MDSRank": { 2024-09-12T02:26:36.389 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-12T02:26:36.389 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-12T02:26:36.389 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 2579, 2024-09-12T02:26:36.389 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.070754253, 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000027434 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: "queue_len": 0, 2024-09-12T02:26:36.390 INFO:teuthology.orchestra.run.smithi003.stdout: "complete_latency": { 2024-09-12T02:26:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.392 INFO:teuthology.orchestra.run.smithi003.stdout: "mds": { 2024-09-12T02:26:36.392 INFO:teuthology.orchestra.run.smithi003.stdout: "request": 0, 2024-09-12T02:26:36.392 INFO:teuthology.orchestra.run.smithi003.stdout: "reply": 0, 2024-09-12T02:26:36.392 INFO:teuthology.orchestra.run.smithi003.stdout: "reply_latency": { 2024-09-12T02:26:36.392 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.393 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.393 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.393 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_reply": 0, 2024-09-12T02:26:36.393 INFO:teuthology.orchestra.run.smithi003.stdout: "forward": 0, 2024-09-12T02:26:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_commit": 0, 2024-09-12T02:26:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_split": 0, 2024-09-12T02:26:36.394 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_merge": 0, 2024-09-12T02:26:36.395 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes": 3, 2024-09-12T02:26:36.395 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_top": 0, 2024-09-12T02:26:36.395 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_bottom": 3, 2024-09-12T02:26:36.395 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:36.395 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_pinned": 3, 2024-09-12T02:26:36.396 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_expired": 0, 2024-09-12T02:26:36.396 INFO:teuthology.orchestra.run.smithi003.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:36.396 INFO:teuthology.orchestra.run.smithi003.stdout: "caps": 0, 2024-09-12T02:26:36.396 INFO:teuthology.orchestra.run.smithi003.stdout: "subtrees": 1, 2024-09-12T02:26:36.396 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_hit": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_forward": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_discover": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:36.397 INFO:teuthology.orchestra.run.smithi003.stdout: "traverse_lock": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "load_cent": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "q": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "exported": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "exported_inodes": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "imported": 0, 2024-09-12T02:26:36.398 INFO:teuthology.orchestra.run.smithi003.stdout: "imported_inodes": 0, 2024-09-12T02:26:36.399 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:36.399 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:36.399 INFO:teuthology.orchestra.run.smithi003.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:36.399 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rfiles": 0, 2024-09-12T02:26:36.399 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rbytes": 0, 2024-09-12T02:26:36.400 INFO:teuthology.orchestra.run.smithi003.stdout: "root_rsnaps": 0, 2024-09-12T02:26:36.400 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:36.400 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:36.400 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:36.400 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:36.401 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:36.402 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:36.402 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:36.402 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:36.402 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:36.402 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps": 0, 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: "process_request_cap_release": 0 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_cache": { 2024-09-12T02:26:36.403 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update": 0, 2024-09-12T02:26:36.404 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:36.404 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_try_discover": 0, 2024-09-12T02:26:36.404 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_send_discover": 0, 2024-09-12T02:26:36.404 INFO:teuthology.orchestra.run.smithi003.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:36.404 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays": 0, 2024-09-12T02:26:36.405 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:36.405 INFO:teuthology.orchestra.run.smithi003.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:36.405 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_created": 0, 2024-09-12T02:26:36.405 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_enqueued": 0, 2024-09-12T02:26:36.405 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: "strays_migrated": 0, 2024-09-12T02:26:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:36.406 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_started": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "recovery_completed": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:36.407 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_flush": 0, 2024-09-12T02:26:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: "ireq_inodestats": 0 2024-09-12T02:26:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.408 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_log": { 2024-09-12T02:26:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: "evlrg": 0, 2024-09-12T02:26:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: "evadd": 0, 2024-09-12T02:26:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: "evex": 0, 2024-09-12T02:26:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: "evtrm": 0, 2024-09-12T02:26:36.409 INFO:teuthology.orchestra.run.smithi003.stdout: "ev": 27, 2024-09-12T02:26:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: "evexg": 0, 2024-09-12T02:26:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: "evexd": 0, 2024-09-12T02:26:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: "segadd": 0, 2024-09-12T02:26:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: "segex": 0, 2024-09-12T02:26:36.410 INFO:teuthology.orchestra.run.smithi003.stdout: "segtrm": 0, 2024-09-12T02:26:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: "seg": 2, 2024-09-12T02:26:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: "segmjr": 2, 2024-09-12T02:26:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: "segexg": 0, 2024-09-12T02:26:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: "segexd": 0, 2024-09-12T02:26:36.411 INFO:teuthology.orchestra.run.smithi003.stdout: "expos": 4194304, 2024-09-12T02:26:36.412 INFO:teuthology.orchestra.run.smithi003.stdout: "wrpos": 4241587, 2024-09-12T02:26:36.412 INFO:teuthology.orchestra.run.smithi003.stdout: "rdpos": 4239936, 2024-09-12T02:26:36.412 INFO:teuthology.orchestra.run.smithi003.stdout: "jlat": { 2024-09-12T02:26:36.412 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.412 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: "replayed": 27 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_mem": { 2024-09-12T02:26:36.413 INFO:teuthology.orchestra.run.smithi003.stdout: "ino": 4, 2024-09-12T02:26:36.414 INFO:teuthology.orchestra.run.smithi003.stdout: "ino+": 24, 2024-09-12T02:26:36.414 INFO:teuthology.orchestra.run.smithi003.stdout: "ino-": 20, 2024-09-12T02:26:36.414 INFO:teuthology.orchestra.run.smithi003.stdout: "dir": 2, 2024-09-12T02:26:36.414 INFO:teuthology.orchestra.run.smithi003.stdout: "dir+": 21, 2024-09-12T02:26:36.414 INFO:teuthology.orchestra.run.smithi003.stdout: "dir-": 19, 2024-09-12T02:26:36.415 INFO:teuthology.orchestra.run.smithi003.stdout: "dn": 3, 2024-09-12T02:26:36.415 INFO:teuthology.orchestra.run.smithi003.stdout: "dn+": 24, 2024-09-12T02:26:36.415 INFO:teuthology.orchestra.run.smithi003.stdout: "dn-": 21, 2024-09-12T02:26:36.415 INFO:teuthology.orchestra.run.smithi003.stdout: "cap": 0, 2024-09-12T02:26:36.415 INFO:teuthology.orchestra.run.smithi003.stdout: "cap+": 0, 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: "cap-": 0, 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: "rss": 49104, 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: "heap": 215328 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_server": { 2024-09-12T02:26:36.416 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_client_request": 0, 2024-09-12T02:26:36.417 INFO:teuthology.orchestra.run.smithi003.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:36.417 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_request": 0, 2024-09-12T02:26:36.417 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_client_session": 0, 2024-09-12T02:26:36.417 INFO:teuthology.orchestra.run.smithi003.stdout: "handle_peer_request": 0, 2024-09-12T02:26:36.417 INFO:teuthology.orchestra.run.smithi003.stdout: "req_create_latency": { 2024-09-12T02:26:36.418 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.418 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.418 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.418 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getattr_latency": { 2024-09-12T02:26:36.419 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.419 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.419 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.419 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:36.420 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.420 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.420 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.420 INFO:teuthology.orchestra.run.smithi003.stdout: "req_link_latency": { 2024-09-12T02:26:36.421 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.421 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.421 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.421 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookup_latency": { 2024-09-12T02:26:36.422 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.422 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.422 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.422 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupino_latency": { 2024-09-12T02:26:36.423 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.424 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.424 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.424 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupname_latency": { 2024-09-12T02:26:36.424 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.425 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.426 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "req_lssnap_latency": { 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mkdir_latency": { 2024-09-12T02:26:36.427 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.428 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.428 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mknod_latency": { 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "req_mksnap_latency": { 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "req_open_latency": { 2024-09-12T02:26:36.429 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "req_readdir_latency": { 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rename_latency": { 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmdir_latency": { 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.431 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.432 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setattr_latency": { 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.433 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setlayout_latency": { 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "req_setxattr_latency": { 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.434 INFO:teuthology.orchestra.run.smithi003.stdout: "req_symlink_latency": { 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "req_unlink_latency": { 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:36.435 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_sessions": { 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "session_count": 1, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "session_add": 2, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "session_remove": 1, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_open": 1, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "sessions_stale": 0, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "total_load": 0, 2024-09-12T02:26:36.436 INFO:teuthology.orchestra.run.smithi003.stdout: "average_load": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "avg_session_uptime": 1567, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:36.437 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:36.438 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:36.439 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 197463, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 54, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:36.440 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 157, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 14160, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 40, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-12T02:26:36.441 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "objecter": { 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_active": 0, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_laggy": 0, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send": 1542, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_send_bytes": 21, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_resend": 3, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_reply": 1542, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "op_latency": { 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1542, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 71.200091888, 2024-09-12T02:26:36.442 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.046173859 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: "op_inflight": 0, 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: "oplen_avg": { 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 1542, 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 1543 2024-09-12T02:26:36.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "op": 1542, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "op_r": 1542, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "op_w": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "op_rmw": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "op_pg": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_stat": 988, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_create": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_read": 552, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_write": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writefull": 0, 2024-09-12T02:26:36.444 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_writesame": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_append": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_zero": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_truncate": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_delete": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_mapext": 0, 2024-09-12T02:26:36.445 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_getxattr": 1, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_setxattr": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:36.446 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_call": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_watch": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_notify": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "osdop_other": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_active": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_send": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_resend": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "linger_ping": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_active": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_send": 0, 2024-09-12T02:26:36.447 INFO:teuthology.orchestra.run.smithi003.stdout: "poolop_resend": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_active": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_send": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "poolstat_resend": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_active": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_send": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "statfs_resend": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "command_active": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "command_send": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "command_resend": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "map_epoch": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "map_full": 0, 2024-09-12T02:26:36.448 INFO:teuthology.orchestra.run.smithi003.stdout: "map_inc": 0, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_sessions": 6, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_open": 6, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_session_close": 0, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_laggy": 0, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_wr": 0, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_rd": 2, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_del": 0 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "oft": { 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_objs": 0, 2024-09-12T02:26:36.449 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_updates": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "omap_total_removes": 0 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "osd-slow-ops": { 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "slow_ops_count": 0 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "purge_queue": { 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:36.450 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_executed": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 1736, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1169121, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 1736, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:36.451 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1736, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1169121, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:36.452 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 1542, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 47288, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1542, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 47288, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 1024, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 1542, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 1542, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 1542, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1542, 2024-09-12T02:26:36.454 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:36.455 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:36.456 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-write_buf_throttle-0x55b3beb3c6e0": { 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 3758096384, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 0, 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:36.457 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:36.458 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:36.458 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:36.458 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.458 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:36.458 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:36.675 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:36 smithi003 ceph-mon[31476]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:36.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:36 smithi026 ceph-mon[35496]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:36.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:36 smithi092 ceph-mon[36089]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:37.292 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-12T02:26:37.292 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-12T02:26:37.292 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-12T02:26:37.292 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T02:26:37.293 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T02:26:37.564 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 180, 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 175, 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 742416, 2024-09-12T02:26:38.067 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 107088, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.031892596, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.012458567, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.023550833, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 174, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.012369744, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000071090 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:38.068 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 742416, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 107088 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 1, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 132, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 160, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 101760, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 3, 2024-09-12T02:26:38.069 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.011008111, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.008010306, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001362014, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 130, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.008123695, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000062489 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:38.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 101760 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 3024, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.002416238, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000036033, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.002354244, 2024-09-12T02:26:38.071 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.004233349, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.001411116 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.072 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:38.073 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:38.074 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:38.075 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148967, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 29, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-09-12T02:26:38.076 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:38.077 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 181, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 717016, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 181, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 181, 2024-09-12T02:26:38.078 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 717016, 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-09-12T02:26:38.079 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-09-12T02:26:38.657 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:38 smithi003 ceph-mon[31476]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:38.686 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T02:26:38.686 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T02:26:38.686 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T02:26:38.686 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T02:26:38.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T02:26:38.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:38 smithi026 ceph-mon[35496]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:38.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:38 smithi092 ceph-mon[36089]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:39.020 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:26:39.716 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 833, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 694, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 286796, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 293842, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.075106277, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.023816650, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.055972955, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.005022595, 2024-09-12T02:26:39.717 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 694, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.045983921, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000066259 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1398, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000276981, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000198 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:39.718 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 32, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 157, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 9562, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 435835, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 16, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.015236001, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.005826054, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.007657033, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000377485, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 156, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.021228847, 2024-09-12T02:26:39.719 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000136082 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000963, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000096 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 306384 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 266, 2024-09-12T02:26:39.720 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 247, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 885982, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 177038, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 10, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.037065977, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.013777830, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.030980727, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000334015, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 244, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.016755841, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000068671 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.721 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 47, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000008872, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000188 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 876784, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 109728 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:39.722 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 76, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.004175482, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000054940 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 16, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000302563, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000018910 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.723 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 8, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 4, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.056378188, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.014094547 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 4, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 11, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-12T02:26:39.724 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 16, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 2, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 14, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 13, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 12, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 1, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 1, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 3, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 24, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 15, 2024-09-12T02:26:39.725 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 4, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 2, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 1, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 1, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 2, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 1, 2024-09-12T02:26:39.726 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 175, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 214153, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 1, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 1, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 5, 2024-09-12T02:26:39.727 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 2, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 7, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 1, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 2, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 2 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 3, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 2, 2024-09-12T02:26:39.728 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 4, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 1, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 1, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-12T02:26:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 5, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 42, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 1, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 16, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-12T02:26:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 16, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 2, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 2, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 2, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 4218926, 2024-09-12T02:26:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 13, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.111868043, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.008605234 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 20, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 27, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 7, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 19, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 26, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 7, 2024-09-12T02:26:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 16, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 31, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 15, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 1, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 5, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 4, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 52376, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 223520 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 11, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 1, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 8, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 7, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 4, 2024-09-12T02:26:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000125991, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000125991 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-12T02:26:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000536367, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000268183 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-12T02:26:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.055715830, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.055715830 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-12T02:26:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 1, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 0, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 2, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 2, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 476, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-12T02:26:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 246103, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 76, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 157, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 81333, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 329, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-12T02:26:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 69, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 36936, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 32, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 69, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 69, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2.020968041, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.029289391 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-12T02:26:39.751 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 69, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 97 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 69, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 2, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 67, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 0, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 10, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 14, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 11, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-12T02:26:39.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 13, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 20, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:39.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-12T02:26:39.754 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 8, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 8, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 23, 2024-09-12T02:26:39.755 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 0, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 4 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 41 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.756 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 1, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 1, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 1, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 1, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1127, 2024-09-12T02:26:39.757 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1066457, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1127, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1127, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1066457, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.758 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 69, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 37037, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 69, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 37037, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.759 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 69, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 69, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 69, 2024-09-12T02:26:39.760 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 69, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 101, 2024-09-12T02:26:39.761 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 101, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:39.762 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x55eea58f4500": { 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 16, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 24622, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 16, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 13, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 24622, 2024-09-12T02:26:39.763 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:39.764 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-12T02:26:40.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:40 smithi026 ceph-mon[35496]: pgmap v584: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:40.670 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-09-12T02:26:40.671 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-12T02:26:40.671 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-09-12T02:26:40.671 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T02:26:40.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T02:26:40.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:40 smithi003 ceph-mon[31476]: pgmap v584: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:40.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:40 smithi092 ceph-mon[36089]: pgmap v584: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 351, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 353, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 25262669, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 111682, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.051148536, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.010775882, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.027148970, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.009550974, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 351, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.030288508, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000086292 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 804, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 782, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 38627659, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 299039, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.106696153, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.023588765, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.069036147, 2024-09-12T02:26:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.018387427, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 781, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.042901794, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000054931 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 869984, 2024-09-12T02:26:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 107280 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 626, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 755, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 21181165, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 504502, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.089874632, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.025671861, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.034394873, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.022461372, 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 754, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.049323265, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000065415 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 307024 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-12T02:26:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2686, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.072174586, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000026870 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 133, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 133, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 133, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-12T02:26:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-09-12T02:26:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 39922, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-12T02:26:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 44, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 5, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 87925693, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 87924042, 2024-09-12T02:26:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 39922 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 4, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 153, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 149, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 2, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 30, 2024-09-12T02:26:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 28, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 133, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 285, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 152, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 54176, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215328 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-09-12T02:26:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-12T02:26:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-12T02:26:41.456 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.457 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.458 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.459 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.460 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.461 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.462 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-09-12T02:26:41.464 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 2, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 1, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3180, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.465 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:41.466 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:41.467 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-12T02:26:41.468 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 214424, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 65, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:41.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 157, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 105680, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 560, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.470 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 1575, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 1575, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1575, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 107.447293864, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.068220504 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-12T02:26:41.471 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1575, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1576 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 1575, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1575, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 915, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 658, 2024-09-12T02:26:41.472 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:41.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-12T02:26:41.474 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-09-12T02:26:41.475 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 8, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 8, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.477 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1777, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 84898307, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1777, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1777, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 84898307, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1575, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 83756253, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1575, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 83756253, 2024-09-12T02:26:41.479 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:41.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1575, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 1575, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1575, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1575, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:41.482 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x5622d879c6e0": { 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:41.483 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:41.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:41.484 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-12T02:26:42.208 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T02:26:42.208 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T02:26:42.208 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T02:26:42.208 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T02:26:42.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T02:26:42.460 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:26:42.748 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:42 smithi026 ceph-mon[35496]: pgmap v585: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-12T02:26:42.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:42 smithi003 ceph-mon[31476]: pgmap v585: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-12T02:26:42.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:42 smithi092 ceph-mon[36089]: pgmap v585: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 11 op/s 2024-09-12T02:26:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-12T02:26:42.975 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 807, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 807, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 4450019, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 255812, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.087212414, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.030173185, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.024904130, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.024571889, 2024-09-12T02:26:42.976 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 806, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.039578197, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000049104 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:42.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 610, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 588, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1206849, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 236606, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.064328422, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019551934, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.034456853, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.014966152, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 587, 2024-09-12T02:26:42.978 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.044188743, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000075278 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 869600, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 106160 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:42.979 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 613, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 742, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 4350575, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 499225, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.063530989, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019706007, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.019133944, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.019051039, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 740, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.027751345, 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000037501 2024-09-12T02:26:42.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 304608 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.981 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3024, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.100129943, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000033111 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-12T02:26:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 757, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 518, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 239, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 16, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-09-12T02:26:42.984 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 2, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-12T02:26:42.985 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 234, 2024-09-12T02:26:42.986 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:42.987 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:42.988 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:26:42.989 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-12T02:26:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 3732, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 2, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 35, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 12811206, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 12817989, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 12816541, 2024-09-12T02:26:42.991 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 3732 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 17, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 785, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 768, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 12, 2024-09-12T02:26:42.992 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 111, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 99, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 757, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 1240, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 483, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 51432, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215328 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-09-12T02:26:42.993 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.995 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-12T02:26:42.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.997 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.998 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:42.999 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.001 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.002 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.003 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.004 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.005 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.006 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3183, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-12T02:26:43.007 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:43.008 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:43.009 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 189423, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 51, 2024-09-12T02:26:43.010 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 157, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:43.011 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 586298, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 3251, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 1825, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 26, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 4, 2024-09-12T02:26:43.012 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 1825, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1825, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 35.838430071, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.019637495 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1825, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1826 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 1825, 2024-09-12T02:26:43.013 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1825, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 1134, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 688, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-09-12T02:26:43.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 2, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:43.015 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-12T02:26:43.016 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 5, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 5, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.017 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:43.018 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 2026, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 9811905, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 2026, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:43.019 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 2026, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 9811905, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:43.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1825, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 8623695, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1825, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 8623695, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1825, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 1825, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1825, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1825, 2024-09-12T02:26:43.022 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x55e8181e66e0": { 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:43.026 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-12T02:26:43.697 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T02:26:43.697 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T02:26:43.697 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T02:26:43.698 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T02:26:43.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T02:26:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:26:44.633 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:44 smithi026 ceph-mon[35496]: pgmap v586: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-12T02:26:44.674 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1205, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1205, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 386197, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 382367, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.112573058, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.036491685, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.030812198, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.035435429, 2024-09-12T02:26:44.675 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1204, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.055098618, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000045762 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:44.676 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 475, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 608, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 117416, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 457728, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.052248829, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.015715035, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.011565251, 2024-09-12T02:26:44.677 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.018066867, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 606, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.032421944, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000053501 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 306928 2024-09-12T02:26:44.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 205, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 183, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 871068, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 109518, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 3, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.030402569, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.011845340, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.026875288, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000021323, 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:44.679 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 182, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.017605123, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000096731 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 870800, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 109200 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.680 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2801, 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.084772731, 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000030265 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2024-09-12T02:26:44.682 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 2, 2024-09-12T02:26:44.683 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 2, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 2, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2024-09-12T02:26:44.684 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:44.685 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:44.686 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:44.688 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2024-09-12T02:26:44.689 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "evlrg": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2024-09-12T02:26:44.690 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 16, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 2, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segmjr": 2, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 4218926, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 4217275, 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2024-09-12T02:26:44.691 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 16 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.692 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 4, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 17, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 13, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 2, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 15, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 13, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 2, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 16, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 14, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2024-09-12T02:26:44.693 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 48392, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 215328 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2024-09-12T02:26:44.694 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.695 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2024-09-12T02:26:44.697 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.698 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.699 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.700 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.701 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.703 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.704 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.705 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2024-09-12T02:26:44.706 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:44.707 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 1, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 0, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2024-09-12T02:26:44.708 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 1839, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:44.709 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:44.710 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:44.711 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 189440, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 52, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:44.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 157, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 13456, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 36, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.713 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 1676, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 1676, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1676, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 11.568815780, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006902634 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.714 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1676, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1677 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 1676, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1676, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 1066, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2024-09-12T02:26:44.715 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 608, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2024-09-12T02:26:44.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2024-09-12T02:26:44.717 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2024-09-12T02:26:44.718 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 5, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 5, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2024-09-12T02:26:44.719 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2024-09-12T02:26:44.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1881, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1191979, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:44.721 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1881, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1881, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1191979, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1676, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 24627, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1676, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 24627, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1676, 2024-09-12T02:26:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 1676, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1676, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1676, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x5575487266e0": { 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2024-09-12T02:26:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-12T02:26:44.728 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-12T02:26:44.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:44 smithi003 ceph-mon[31476]: pgmap v586: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-12T02:26:44.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:44 smithi092 ceph-mon[36089]: pgmap v586: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-12T02:26:45.576 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-12T02:26:45.576 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-12T02:26:45.576 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-12T02:26:45.576 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T02:26:45.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T02:26:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:26:46.506 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:46 smithi092 ceph-mon[36089]: pgmap v587: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-12T02:26:46.596 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 37, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 41, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 8521, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 60633, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 13, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.012035783, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.001937815, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.006737750, 2024-09-12T02:26:46.597 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.001084704, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.019618929, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000530241 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 31, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000004955, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000159 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T02:26:46.598 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 668, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 726, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 229418, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 545041, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 7, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.075788727, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.033204669, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.045616889, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.003306178, 2024-09-12T02:26:46.599 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 726, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.049889977, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000068718 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1189, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000233163, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000196 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 308192 2024-09-12T02:26:46.600 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 260, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 231, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 894565, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 244577, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 15, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.036402503, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.011954511, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.031903795, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000555152, 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:46.601 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 231, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026611264, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000115200 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000929, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000092 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 890896, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 113344 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.602 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001281094, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000034624 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:46.603 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000314464, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000157232 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 7, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.604 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 7, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 11, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 15, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 15, 2024-09-12T02:26:46.605 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 13, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 6, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 18, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 11, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 6, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 1, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-12T02:26:46.606 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 1, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 175, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 214153, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-12T02:26:46.607 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:46.608 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 3, 2024-09-12T02:26:46.609 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 2, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 3, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:46.610 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 5, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 30, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:46.611 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 3, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 3, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 2, 2024-09-12T02:26:46.612 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4195200, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-09-12T02:26:46.613 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002850713, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001425356 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 19, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 22, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 3, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 18, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 22, 2024-09-12T02:26:46.614 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 4, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 15, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 21, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 6, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 51492, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223520 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 7, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 7, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 7, 2024-09-12T02:26:46.615 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.616 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.617 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.618 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.619 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.620 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-12T02:26:46.621 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.622 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.623 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.624 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.625 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.626 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.627 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 1, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 481, 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:46.628 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:46.629 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:46.630 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 230135, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 74, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:46.631 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:46.632 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 76701, 2024-09-12T02:26:46.633 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 305, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 37, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 11108, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-09-12T02:26:46.634 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 37, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.216993527, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005864689 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37, 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 59 2024-09-12T02:26:46.635 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 37, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 37, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 2, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 6, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-12T02:26:46.636 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 6, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:46.637 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-12T02:26:46.638 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 83, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 1, 2024-09-12T02:26:46.639 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 9, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 9, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 14, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 1 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-12T02:26:46.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 29 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:46.641 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 961, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1031063, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 961, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 961, 2024-09-12T02:26:46.642 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1031063, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:46.643 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 37, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 11108, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 37, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 11108, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.644 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.645 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.645 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.645 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:46.645 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:46.645 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 37, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 37, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 37, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 37, 2024-09-12T02:26:46.646 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:46.647 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x559073128500": { 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:46.648 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 3, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 896, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 3, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 896, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:46.649 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.650 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:46.650 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-12T02:26:46.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:46 smithi026 ceph-mon[35496]: pgmap v587: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-12T02:26:46.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:46 smithi003 ceph-mon[31476]: pgmap v587: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-09-12T02:26:47.595 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-12T02:26:47.596 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-12T02:26:47.596 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-12T02:26:47.596 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T02:26:47.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T02:26:47.843 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:26:48.396 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-12T02:26:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:48.396 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 42726, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 45592, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 14687906, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 40243602, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 14, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 2.345545059, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.876308037, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 2.943840727, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.091431045, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 45591, 2024-09-12T02:26:48.397 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 16.345406768, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000358522 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 66464, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.026481681, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000398 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.398 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2497, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2641, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 663977, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 40423546, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.368747820, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.191100542, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.059770569, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.099693323, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2639, 2024-09-12T02:26:48.399 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.326306915, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000123647 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 126, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000019155, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000152 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3536, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 312960 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:48.400 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2949, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2715, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1707486, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5438292, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 15, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 8, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.300558625, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.142112816, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.120888208, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.049219015, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2714, 2024-09-12T02:26:48.401 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.189592878, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000069857 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2273, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000415027, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000182 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 883776, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 120528 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-12T02:26:48.402 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4603, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 14.237206627, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003093027 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.403 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3422, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.098197439, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000028695 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 669, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 655, 2024-09-12T02:26:48.404 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 655, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.782471543, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004248048 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 14, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:48.405 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 20, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 1, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 16, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 1, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 1, 2024-09-12T02:26:48.406 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 14, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 14, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 21, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 1, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 1, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-09-12T02:26:48.407 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 846, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 815, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 10, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 6, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 4, 2024-09-12T02:26:48.408 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 3, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 4, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 29, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 2, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 175, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 214153, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 2, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:48.409 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 1, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 1, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 1, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 6, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 584, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 3020, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 39649, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 38669, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 27, 2024-09-12T02:26:48.410 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 156 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 3, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 2, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 7, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 2, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 132, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 132, 2024-09-12T02:26:48.411 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 5, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 451, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 4, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-12T02:26:48.412 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 1, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 39922, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 39922, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 44, 2024-09-12T02:26:48.413 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 44, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 5, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 87925693, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2697, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 350.759007917, 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.130055249 2024-09-12T02:26:48.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 20, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 163, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 143, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 19, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 41, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 22, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 16, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 300, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 284, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 1, 2024-09-12T02:26:48.415 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 476, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 475, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 68256, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223520 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 720, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 5, 2024-09-12T02:26:48.416 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 669, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 12, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 20, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 105, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.170664242, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001625373 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002877859, 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000959286 2024-09-12T02:26:48.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 142, 2024-09-12T02:26:48.418 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.074337731, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000523505 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.419 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:48.420 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.421 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 252, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.303288560, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.009140033 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 21, 2024-09-12T02:26:48.423 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.015109535, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000719501 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.091584600, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.030528200 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.425 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.426 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.427 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-12T02:26:48.428 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 129, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.124609016, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000965961 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.429 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 309, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 309, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 506, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:48.430 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:48.431 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:48.432 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:48.433 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:48.434 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 295511, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 105, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:48.435 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:48.436 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-09-12T02:26:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:48.437 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 81973, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 369, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-12T02:26:48.438 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-12T02:26:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-12T02:26:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-12T02:26:48.439 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-12T02:26:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 6240, 2024-09-12T02:26:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 83968512, 2024-09-12T02:26:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-09-12T02:26:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 6240, 2024-09-12T02:26:48.440 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6240, 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 497.071035785, 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.079658819 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-12T02:26:48.441 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-12T02:26:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6240, 2024-09-12T02:26:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 6514 2024-09-12T02:26:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 6240, 2024-09-12T02:26:48.442 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 8, 2024-09-12T02:26:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 6232, 2024-09-12T02:26:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-12T02:26:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-12T02:26:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-09-12T02:26:48.443 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-09-12T02:26:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 6, 2024-09-12T02:26:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 2703, 2024-09-12T02:26:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 33, 2024-09-12T02:26:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-12T02:26:48.444 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 1, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 3243, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:48.445 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 2, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:48.446 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:48.447 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-12T02:26:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-12T02:26:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-12T02:26:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-12T02:26:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-12T02:26:48.448 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-12T02:26:48.449 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 346, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-09-12T02:26:48.450 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 150 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 450 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:48.451 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 26, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 2, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 3212, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 132, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:48.452 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 48168, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 12708837, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 48168, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 48168, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 12708837, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.453 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 6240, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 83981852, 2024-09-12T02:26:48.454 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 6240, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 83981852, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:48.455 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 6240, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 6240, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 6240, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 6240, 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:48.456 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 132, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 13340, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 132, 2024-09-12T02:26:48.457 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 6, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 13340, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55d5651c0500": { 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:48.458 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 39922, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 83731389, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 39922, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2697, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 83731389, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:48.459 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.460 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:48.460 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-12T02:26:48.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:48 smithi026 ceph-mon[35496]: pgmap v588: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:48.771 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:48 smithi092 ceph-mon[36089]: pgmap v588: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:48.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:48 smithi003 ceph-mon[31476]: pgmap v588: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-12T02:26:49.218 INFO:tasks.check_counter:Daemon mds.f mds.exported=4 2024-09-12T02:26:49.218 INFO:tasks.check_counter:Daemon mds.f mds.imported=4 2024-09-12T02:26:49.218 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=1 2024-09-12T02:26:49.218 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T02:26:49.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T02:26:49.465 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:26:49.987 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-12T02:26:49.987 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:49.987 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 6952, 2024-09-12T02:26:49.987 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 9715, 2024-09-12T02:26:49.987 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 2331424, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 14589171, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 53, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.513236100, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.182540124, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.354043224, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.074797775, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 9711, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.690148022, 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000071068 2024-09-12T02:26:49.988 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 12172, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002666002, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000219 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 57824, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 30736 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2349, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2532, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 1689071, 2024-09-12T02:26:49.989 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 1628751, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 10, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.216719042, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.090636493, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.105872779, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.027053618, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2534, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.590805633, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000627784 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:49.990 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2201, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000562774, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000255 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 894192, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 116576 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2190, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 2567, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 771782, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 2385528, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 88, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-09-12T02:26:49.991 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.220038993, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.074904639, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.127587759, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.022994147, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2565, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.044113512, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000407061 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3026, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000761267, 2024-09-12T02:26:49.992 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000251 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 36160, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 315408 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1956, 2024-09-12T02:26:49.993 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.800893540, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000920702 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1420, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.081461280, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000057367 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.994 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 2363, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 2343, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2343, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 12.469410251, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005321984 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 20, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 3, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 2, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 398, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-12T02:26:49.995 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 30, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 5, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 13, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 12, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 22, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 40, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 2, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 2, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 3, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 3144, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 2993, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 12, 2024-09-12T02:26:49.996 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 50, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 5, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 33, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 5, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 2, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 296, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:49.997 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 4, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 234, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 296, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 110, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 4, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 106, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 186, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 20, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 605, 2024-09-12T02:26:49.998 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 2060, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 6, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 1731, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 721, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 130, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 1333 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 4, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-09-12T02:26:49.999 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 20, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 799, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 738, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 61, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 5, 2024-09-12T02:26:50.000 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 865, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 4, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 5, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 3732, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 3724, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 3724, 2024-09-12T02:26:50.001 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 8, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 622, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 39, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 38, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 38, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 1, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 1, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 2, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 12811206, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 12817989, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-09-12T02:26:50.002 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 654, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 17.628359001, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.026954677 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 24, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 800, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 776, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 20, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 129, 2024-09-12T02:26:50.003 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 109, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 30, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1349, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 1319, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 2, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 1038, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 1036, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 63528, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 231712 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 3074, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 12, 2024-09-12T02:26:50.004 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 2363, 2024-09-12T02:26:50.005 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 22, 2024-09-12T02:26:50.005 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 36, 2024-09-12T02:26:50.005 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-12T02:26:50.005 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 660, 2024-09-12T02:26:50.008 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.655521845, 2024-09-12T02:26:50.008 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000993214 2024-09-12T02:26:50.008 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 82, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.013952686, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000170154 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 61, 2024-09-12T02:26:50.009 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.046101303, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000755759 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 246, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.075084982, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000305223 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-12T02:26:50.010 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000140629, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000140629 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.011 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-12T02:26:50.012 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 105, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.094323276, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000898316 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-12T02:26:50.013 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 49, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.009002951, 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000183733 2024-09-12T02:26:50.014 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 228, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.846294628, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003711818 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 113, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 9.028231428, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.079895853 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.015 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 96, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.787559080, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.008203740 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.016 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.032373958, 2024-09-12T02:26:50.017 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004624851 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001118293, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000279573 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-12T02:26:50.018 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 8, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.018999573, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002374946 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 13, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.008434078, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000648775 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.019 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 670, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.852271541, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001272047 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.020 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 3, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 543, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 543, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 505, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-12T02:26:50.021 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 120, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 120, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:50.022 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:50.023 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 302775, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 106, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:50.024 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 106658, 2024-09-12T02:26:50.025 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 763, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 3211, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 9565565, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 32, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 3211, 2024-09-12T02:26:50.026 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3211, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 125.528226345, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.039093187 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3211, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4663 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 3211, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 337, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 2874, 2024-09-12T02:26:50.027 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 300, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 322, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 32, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 682, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 133, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 1254, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-12T02:26:50.028 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 305, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 644, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-12T02:26:50.029 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 6, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-12T02:26:50.030 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 83, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 2, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 881, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 10, 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 94 2024-09-12T02:26:50.031 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 859 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:50.032 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 738, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 738, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 11487, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 3733344, 2024-09-12T02:26:50.033 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 11487, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 11487, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 3733344, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.034 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3211, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 9640047, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3211, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9640047, 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:50.035 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:50.036 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3211, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 3211, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3211, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 3211, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:50.037 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 738, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 74538, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 738, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 28, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 74538, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.038 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55e8612a8500": { 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 3732, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 8623685, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 3732, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 654, 2024-09-12T02:26:50.039 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 8623685, 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:50.040 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-12T02:26:50.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:50 smithi026 ceph-mon[35496]: pgmap v589: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:50.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:50 smithi003 ceph-mon[31476]: pgmap v589: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:50.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:50 smithi092 ceph-mon[36089]: pgmap v589: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:26:50.852 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-09-12T02:26:50.852 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-09-12T02:26:50.852 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-09-12T02:26:50.852 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T02:26:50.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T02:26:51.099 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:26:51.648 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-09-12T02:26:51.648 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1630, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1630, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 466231, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 517523, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.137934920, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.041572417, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.036985623, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.049391116, 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:51.649 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1629, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.078811682, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000048380 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 209, 2024-09-12T02:26:51.650 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 187, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 872172, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 111818, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 3, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.029730491, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.011785254, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.025982647, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000077406, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 186, 2024-09-12T02:26:51.651 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.010742651, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000057756 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 871616, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 111120 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 2, 2024-09-12T02:26:51.652 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 137, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 428, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 309214, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 4, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 3, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.007973439, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.005505022, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.001286725, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.000027630, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 135, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.008578091, 2024-09-12T02:26:51.653 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000063541 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 308896 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-09-12T02:26:51.654 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2722, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.072951671, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000026800 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.655 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.656 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-09-12T02:26:51.657 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 1, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-09-12T02:26:51.658 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T02:26:51.659 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-09-12T02:26:51.660 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-09-12T02:26:51.661 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-09-12T02:26:51.662 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 3, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-09-12T02:26:51.663 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 2, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 4195200, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 4194892, 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-09-12T02:26:51.664 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 3 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 3, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 0, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 1, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 1, 2024-09-12T02:26:51.665 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 48620, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215328 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-09-12T02:26:51.666 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-09-12T02:26:51.667 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.668 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.669 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.670 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.671 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.672 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-09-12T02:26:51.673 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.674 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.675 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.676 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.677 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.678 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.679 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-09-12T02:26:51.680 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 1, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 1816, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-09-12T02:26:51.681 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-09-12T02:26:51.682 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-09-12T02:26:51.683 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 181281, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 46, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-09-12T02:26:51.684 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 7600, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 157, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-09-12T02:26:51.685 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 8512, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 13, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-09-12T02:26:51.686 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 1629, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 1629, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1629, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 57.392271290, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.035231596 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-09-12T02:26:51.687 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1629, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1630 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 1629, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 1629, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1040, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 587, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-09-12T02:26:51.688 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-09-12T02:26:51.689 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-09-12T02:26:51.690 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-09-12T02:26:51.691 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 4, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 4, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-09-12T02:26:51.692 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.693 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 1837, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 1159933, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 1837, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1837, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1159933, 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:51.694 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:51.695 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1629, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 901, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1629, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 901, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-09-12T02:26:51.696 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 1629, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 1629, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1629, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 1629, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.697 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-12T02:26:51.698 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55896f74a6e0": { 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-09-12T02:26:51.699 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-09-12T02:26:51.700 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-09-12T02:26:52.441 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T02:26:52.441 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T02:26:52.442 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-12T02:26:52.442 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T02:26:52.452 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T02:26:52.452 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:26:52.452 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:26:52.483 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:26:52.483 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:26:52.543 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T02:26:52.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:52 smithi026 ceph-mon[35496]: pgmap v590: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-12T02:26:52.777 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:52 smithi003 ceph-mon[31476]: pgmap v590: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-12T02:26:52.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:52 smithi092 ceph-mon[36089]: pgmap v590: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-12T02:26:52.856 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 -- 172.21.15.3:0/1299439150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4102790 msgr2=0x7f6bc41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:53.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 --2- 172.21.15.3:0/1299439150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4102790 0x7f6bc41067e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009900 tx=0x7f6bac02a410 comp rx=0 tx=0).stop 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 -- 172.21.15.3:0/1299439150 shutdown_connections 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 --2- 172.21.15.3:0/1299439150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4106f10 0x7f6bc410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 --2- 172.21.15.3:0/1299439150 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4102790 0x7f6bc41067e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 --2- 172.21.15.3:0/1299439150 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4101de0 0x7f6bc41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 -- 172.21.15.3:0/1299439150 >> 172.21.15.3:0/1299439150 conn(0x7f6bc40fd8c0 msgr2=0x7f6bc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 -- 172.21.15.3:0/1299439150 shutdown_connections 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.347+0000 7f6bc8931640 1 -- 172.21.15.3:0/1299439150 wait complete. 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.348+0000 7f6bc8931640 1 Processor -- start 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.348+0000 7f6bc8931640 1 -- start start 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.348+0000 7f6bc8931640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.348+0000 7f6bc8931640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4102790 0x7f6bc41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:53.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111f70 con 0x7f6bc4106f10 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4111df0 con 0x7f6bc4101de0 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41120f0 con 0x7f6bc4102790 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51948/0 (socket says 172.21.15.3:51948) 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 -- 172.21.15.3:0/685805045 learned_addr learned my addr 172.21.15.3:0/685805045 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bb9d9b640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4102790 0x7f6bc41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc259c640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 -- 172.21.15.3:0/685805045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4102790 msgr2=0x7f6bc41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4102790 0x7f6bc41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 -- 172.21.15.3:0/685805045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4101de0 msgr2=0x7f6bc4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 -- 172.21.15.3:0/685805045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419d440 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc2d9d640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00cdb0 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc005020 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419d730 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0116b0 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.349+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a5010 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.350+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc41021c0 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.352+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6bbc005560 con 0x7f6bc4106f10 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.353+0000 7f6bbbfff640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 0x7f6bb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.353+0000 7f6bc259c640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 0x7f6bb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:53.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.353+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f6bbc0cf610 con 0x7f6bc4106f10 2024-09-12T02:26:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.354+0000 7f6bc259c640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 0x7f6bb00a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005fd0 tx=0x7f6bb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:53.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.356+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bbc016030 con 0x7f6bc4106f10 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.545+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6bc410a150 con 0x7f6bc4106f10 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.546+0000 7f6bbbfff640 1 -- 172.21.15.3:0/685805045 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+580 (secure 0 0 0) 0x7f6bbc09c560 con 0x7f6bc4106f10 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2921615271 2024-09-13T02:09:36.346826+0000 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2551485583 2024-09-13T02:09:36.346826+0000 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/1539065682 2024-09-13T02:09:36.346826+0000 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2796707951 2024-09-13T02:09:12.115573+0000 2024-09-12T02:26:53.546 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1673598194 2024-09-13T02:09:36.346826+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/814335086 2024-09-13T02:09:12.115573+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/1539065682 2024-09-13T02:09:36.346826+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/814335086 2024-09-13T02:09:12.115573+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2395673224 2024-09-13T02:09:12.115573+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3787188077 2024-09-13T02:09:12.115573+0000 2024-09-12T02:26:53.547 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 msgr2=0x7f6bb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 0x7f6bb00a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005fd0 tx=0x7f6bb4005850 comp rx=0 tx=0).stop 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 msgr2=0x7f6bc419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00eaa0 tx=0x7f6bbc00ef60 comp rx=0 tx=0).stop 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 shutdown_connections 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f6bb00a2b80 0x7f6bb00a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6bc4106f10 0x7f6bc419cd60 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6bc4102790 0x7f6bc41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 --2- 172.21.15.3:0/685805045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bc4101de0 0x7f6bc4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 >> 172.21.15.3:0/685805045 conn(0x7f6bc40fd8c0 msgr2=0x7f6bc40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 shutdown_connections 2024-09-12T02:26:53.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:53.548+0000 7f6bc8931640 1 -- 172.21.15.3:0/685805045 wait complete. 2024-09-12T02:26:54.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T02:26:54.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T02:26:54.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:26:54.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.356 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T02:26:54.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.427 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T02:26:54.499 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T02:26:54.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.499 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:26:54.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:54 smithi003 ceph-mon[31476]: pgmap v591: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:54.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:54 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/685805045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:26:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.571 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-12T02:26:54.690 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T02:26:54.690 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-09-12T02:26:54.690 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:26:54.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=639 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/4b9ad3227288e24a40293073134a4e15b1c01d22d51be17e2a695968880fc6bd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/4b9ad3227288e24a40293073134a4e15b1c01d22d51be17e2a695968880fc6bd/diff,workdir=/var/lib/containers/storage/overlay/4b9ad3227288e24a40293073134a4e15b1c01d22d51be17e2a695968880fc6bd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/822773debab69039d968f826a1f302ebcd1325380ee71fba052e05968a4f513b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/822773debab69039d968f826a1f302ebcd1325380ee71fba052e05968a4f513b/diff,workdir=/var/lib/containers/storage/overlay/822773debab69039d968f826a1f302ebcd1325380ee71fba052e05968a4f513b/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/047534b80bbd4f9c23d0b7554291d9a1714bdade72b18461ace28c3b0968ec2d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/047534b80bbd4f9c23d0b7554291d9a1714bdade72b18461ace28c3b0968ec2d/diff,workdir=/var/lib/containers/storage/overlay/047534b80bbd4f9c23d0b7554291d9a1714bdade72b18461ace28c3b0968ec2d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1d3a41b9b104b8ebb8dcf3c87a13a1535611c67c8a5f3f437ec4337881295779/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/1d3a41b9b104b8ebb8dcf3c87a13a1535611c67c8a5f3f437ec4337881295779/diff,workdir=/var/lib/containers/storage/overlay/1d3a41b9b104b8ebb8dcf3c87a13a1535611c67c8a5f3f437ec4337881295779/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8e8c1de0f25d7b92c600dd6a06b8142b77e8cc7d6ea7d497ffeba7fb220c8ba3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/8e8c1de0f25d7b92c600dd6a06b8142b77e8cc7d6ea7d497ffeba7fb220c8ba3/diff,workdir=/var/lib/containers/storage/overlay/8e8c1de0f25d7b92c600dd6a06b8142b77e8cc7d6ea7d497ffeba7fb220c8ba3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/94bd71867904787eefa8d083f354c908314cdc0a8d8656a768c5e008aafd7088/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/94bd71867904787eefa8d083f354c908314cdc0a8d8656a768c5e008aafd7088/diff,workdir=/var/lib/containers/storage/overlay/94bd71867904787eefa8d083f354c908314cdc0a8d8656a768c5e008aafd7088/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3c2962acee8410179ec29168a706094b5d1beff5c10ee15dfa045f71ce212b59/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/3c2962acee8410179ec29168a706094b5d1beff5c10ee15dfa045f71ce212b59/diff,workdir=/var/lib/containers/storage/overlay/3c2962acee8410179ec29168a706094b5d1beff5c10ee15dfa045f71ce212b59/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a137ef84681685cb106923aa17d2b0fbe2fff617218b4f5c3375a994af4dd22a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/a137ef84681685cb106923aa17d2b0fbe2fff617218b4f5c3375a994af4dd22a/diff,workdir=/var/lib/containers/storage/overlay/a137ef84681685cb106923aa17d2b0fbe2fff617218b4f5c3375a994af4dd22a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8b15d76c5cb624adc764b9a44f8722149ecaea201c62eba37f1e6298e0643bb7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/8b15d76c5cb624adc764b9a44f8722149ecaea201c62eba37f1e6298e0643bb7/diff,workdir=/var/lib/containers/storage/overlay/8b15d76c5cb624adc764b9a44f8722149ecaea201c62eba37f1e6298e0643bb7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/867fa1f8e4a9b7a7db20794934e839fa76052e1e0d704ede3b0210ec0d37b71a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RBRS2WYDXALK2ZGPZRUXMPPTI4:/var/lib/containers/storage/overlay/l/LH4MIRDCO74MXUFPBCWCLJNDWB,upperdir=/var/lib/containers/storage/overlay/867fa1f8e4a9b7a7db20794934e839fa76052e1e0d704ede3b0210ec0d37b71a/diff,workdir=/var/lib/containers/storage/overlay/867fa1f8e4a9b7a7db20794934e839fa76052e1e0d704ede3b0210ec0d37b71a/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 2024-09-12T02:26:54.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.714 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-09-12T02:26:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:54 smithi026 ceph-mon[35496]: pgmap v591: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:54 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/685805045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:26:54.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:26:54.771 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-09-12T02:26:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:54 smithi092 ceph-mon[36089]: pgmap v591: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:54.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:54 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/685805045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T02:26:54.894 INFO:teuthology.orchestra.run.smithi003.stderr:Cannot find device "ceph-brx" 2024-09-12T02:26:54.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:26:54.896 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T02:26:54.906 INFO:tasks.cephadm:Teardown begin 2024-09-12T02:26:54.907 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T02:26:54.907 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:26:55.164 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:55.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 -- 172.21.15.3:0/2691285368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102ef0 msgr2=0x7fdcc010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2691285368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102ef0 0x7fdcc010da40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009ca0 tx=0x7fdcb802a950 comp rx=0 tx=0).stop 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 -- 172.21.15.3:0/2691285368 shutdown_connections 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2691285368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102ef0 0x7fdcc010da40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2691285368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0102570 0x7fdcc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.649+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2691285368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0100e20 0x7fdcc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- 172.21.15.3:0/2691285368 >> 172.21.15.3:0/2691285368 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- 172.21.15.3:0/2691285368 shutdown_connections 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- 172.21.15.3:0/2691285368 wait complete. 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 Processor -- start 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- start start 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0100e20 0x7fdcc00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:55.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 0x7fdcc00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0110fd0 con 0x7fdcc0102570 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0110e50 con 0x7fdcc0100e20 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.650+0000 7fdcc532b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111150 con 0x7fdcc0102ef0 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51968/0 (socket says 172.21.15.3:51968) 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 -- 172.21.15.3:0/2780475990 learned_addr learned my addr 172.21.15.3:0/2780475990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbeffd640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0100e20 0x7fdcc00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbf7fe640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 0x7fdcc00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 -- 172.21.15.3:0/2780475990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 msgr2=0x7fdcc00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 0x7fdcc00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 -- 172.21.15.3:0/2780475990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0100e20 msgr2=0x7fdcc00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0100e20 0x7fdcc00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 -- 172.21.15.3:0/2780475990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a6960 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbf7fe640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 0x7fdcc00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcb7fff640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e9f0 tx=0x7fdca800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca800cd80 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a6c50 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca8010910 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80188f0 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.651+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a7190 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.652+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0101200 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.654+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdca8010ac0 con 0x7fdcc0102570 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.655+0000 7fdcbcff9640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 0x7fdcac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:55.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.655+0000 7fdcbeffd640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 0x7fdcac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.655+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fdca80cf4a0 con 0x7fdcc0102570 2024-09-12T02:26:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.656+0000 7fdcbeffd640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 0x7fdcac0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00095f0 tx=0x7fdcb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:55.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.659+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdca809c390 con 0x7fdcc0102570 2024-09-12T02:26:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.872+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc010a0a0 con 0x7fdcc0102570 2024-09-12T02:26:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.873+0000 7fdcbcff9640 1 -- 172.21.15.3:0/2780475990 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7fdca80b3ed0 con 0x7fdcc0102570 2024-09-12T02:26:55.874 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:55.874 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"btime":"2024-09-12T02:26:08:004227+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:26:07.197355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:26:55.874 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.875+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 msgr2=0x7fdcac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 0x7fdcac0a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdcb00095f0 tx=0x7fdcb0005f90 comp rx=0 tx=0).stop 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 msgr2=0x7fdcc00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdca800e9f0 tx=0x7fdca800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 shutdown_connections 2024-09-12T02:26:55.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fdcac0a2b80 0x7fdcac0a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcc0102ef0 0x7fdcc00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdcc0102570 0x7fdcc00fd1c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 --2- 172.21.15.3:0/2780475990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc0100e20 0x7fdcc00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 >> 172.21.15.3:0/2780475990 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 shutdown_connections 2024-09-12T02:26:55.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:55.876+0000 7fdcc532b640 1 -- 172.21.15.3:0/2780475990 wait complete. 2024-09-12T02:26:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:56 smithi003 ceph-mon[31476]: pgmap v592: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:56 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2780475990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:56 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.578 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:26:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:56 smithi026 ceph-mon[35496]: pgmap v592: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:56 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2780475990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:56 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:56 smithi092 ceph-mon[36089]: pgmap v592: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-12T02:26:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:56 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2780475990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:56 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:26:56.853 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 -- 172.21.15.3:0/2298776069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac10c170 msgr2=0x7fb6ac10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 --2- 172.21.15.3:0/2298776069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac10c170 0x7fb6ac10e610 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009ca0 tx=0x7fb6a402a950 comp rx=0 tx=0).stop 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 -- 172.21.15.3:0/2298776069 shutdown_connections 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 --2- 172.21.15.3:0/2298776069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac10c170 0x7fb6ac10e610 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 --2- 172.21.15.3:0/2298776069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 --2- 172.21.15.3:0/2298776069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac102d40 0x7fb6ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 -- 172.21.15.3:0/2298776069 >> 172.21.15.3:0/2298776069 conn(0x7fb6ac0fc7a0 msgr2=0x7fb6ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 -- 172.21.15.3:0/2298776069 shutdown_connections 2024-09-12T02:26:57.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.375+0000 7fb6b2928640 1 -- 172.21.15.3:0/2298776069 wait complete. 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 Processor -- start 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 -- start start 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac10c170 0x7fb6ac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac111f70 con 0x7fb6ac102d40 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac111df0 con 0x7fb6ac103660 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6b2928640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1120f0 con 0x7fb6ac10c170 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6abfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51986/0 (socket says 172.21.15.3:51986) 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6abfff640 1 -- 172.21.15.3:0/4207604215 learned_addr learned my addr 172.21.15.3:0/4207604215 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.376+0000 7fb6ab7fe640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:57.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6b0ec5640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac10c170 0x7fb6ac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 -- 172.21.15.3:0/4207604215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac10c170 msgr2=0x7fb6ac19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac10c170 0x7fb6ac19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 -- 172.21.15.3:0/4207604215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 msgr2=0x7fb6ac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 -- 172.21.15.3:0/4207604215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac19d3c0 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6ab7fe640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6abfff640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00d770 tx=0x7fb69c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c017070 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb69c005500 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c004e00 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac19d6b0 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.377+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1a4f90 con 0x7fb6ac102d40 2024-09-12T02:26:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.378+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac103120 con 0x7fb6ac102d40 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.379+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb69c004fa0 con 0x7fb6ac102d40 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.380+0000 7fb6a97fa640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 0x7fb6940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.380+0000 7fb6ab7fe640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 0x7fb6940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.380+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7fb69c0ceeb0 con 0x7fb6ac102d40 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.381+0000 7fb6ab7fe640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 0x7fb6940a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00095f0 tx=0x7fb6a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:57.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.384+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb69c09b750 con 0x7fb6ac102d40 2024-09-12T02:26:57.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.597+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6ac1a5160 con 0x7fb6ac102d40 2024-09-12T02:26:57.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:26:57.598 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:26:57.598 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:57 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:26:57.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.598+0000 7fb6a97fa640 1 -- 172.21.15.3:0/4207604215 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7fb69c09fa50 con 0x7fb6ac102d40 2024-09-12T02:26:57.598 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:57.599 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":41,"btime":"2024-09-12T02:26:08:004227+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/3477371464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":3477371464},{"type":"v1","addr":"172.21.15.3:6835","nonce":3477371464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/4187721968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":4187721968},{"type":"v1","addr":"172.21.15.3:6837","nonce":4187721968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:16:29.718750+0000","modified":"2024-09-12T02:26:07.197355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24508,"mds_1":24493,"mds_2":24487,"mds_3":24482,"mds_4":14619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14613":{"gid":14613,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.3:6839/775355044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":775355044},{"type":"v1","addr":"172.21.15.3:6839","nonce":775355044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14619":{"gid":14619,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":7,"addr":"172.21.15.3:6841/3157578506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":3157578506},{"type":"v1","addr":"172.21.15.3:6841","nonce":3157578506}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6833/1804609961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1804609961},{"type":"v1","addr":"172.21.15.26:6833","nonce":1804609961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1866420590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1866420590},{"type":"v1","addr":"172.21.15.26:6835","nonce":1866420590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/3877701831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":3877701831},{"type":"v1","addr":"172.21.15.92:6833","nonce":3877701831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6837/2891911183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":2891911183},{"type":"v1","addr":"172.21.15.26:6837","nonce":2891911183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6839/1997151172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1997151172},{"type":"v1","addr":"172.21.15.26:6839","nonce":1997151172}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":60,"addr":"172.21.15.92:6835/1180218932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1180218932},{"type":"v1","addr":"172.21.15.92:6835","nonce":1180218932}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.92:6837/487477954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":487477954},{"type":"v1","addr":"172.21.15.92:6837","nonce":487477954}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.92:6839/1571545450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":1571545450},{"type":"v1","addr":"172.21.15.92:6839","nonce":1571545450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24508,"qdb_cluster":[24493,24487,24508,24482,14619]},"id":1}]} 2024-09-12T02:26:57.599 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 41 2024-09-12T02:26:57.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 msgr2=0x7fb6940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:57.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 0x7fb6940a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00095f0 tx=0x7fb6a0005f90 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 msgr2=0x7fb6ac198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00d770 tx=0x7fb69c00dc30 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 shutdown_connections 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7fb6940a2b80 0x7fb6940a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6ac10c170 0x7fb6ac19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6ac103660 0x7fb6ac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 --2- 172.21.15.3:0/4207604215 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac102d40 0x7fb6ac198410 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 >> 172.21.15.3:0/4207604215 conn(0x7fb6ac0fc7a0 msgr2=0x7fb6ac0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 shutdown_connections 2024-09-12T02:26:57.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:57.600+0000 7fb6b2928640 1 -- 172.21.15.3:0/4207604215 wait complete. 2024-09-12T02:26:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:26:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:26:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:57 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:26:57.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:26:57.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:26:57.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:57 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' 2024-09-12T02:26:58.409 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:26:58.674 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:26:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:58 smithi026 ceph-mon[35496]: pgmap v593: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:58.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:58 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/4207604215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:58 smithi003 ceph-mon[31476]: pgmap v593: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:58.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:58 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/4207604215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:58.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:58 smithi092 ceph-mon[36089]: pgmap v593: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-12T02:26:58.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:58 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/4207604215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- 172.21.15.3:0/4153117092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e41082a0 msgr2=0x7f42e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/4153117092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e41082a0 0x7f42e410ec40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009cd0 tx=0x7f42dc02a910 comp rx=0 tx=0).stop 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- 172.21.15.3:0/4153117092 shutdown_connections 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/4153117092 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e41082a0 0x7f42e410ec40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/4153117092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41036e0 0x7f42e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/4153117092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e4102d30 0x7f42e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- 172.21.15.3:0/4153117092 >> 172.21.15.3:0/4153117092 conn(0x7f42e40fc7d0 msgr2=0x7f42e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- 172.21.15.3:0/4153117092 shutdown_connections 2024-09-12T02:26:59.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- 172.21.15.3:0/4153117092 wait complete. 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 Processor -- start 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.168+0000 7f42ea5f9640 1 -- start start 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e41036e0 0x7f42e4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41082a0 0x7f42e419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4112160 con 0x7f42e4102d30 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111fe0 con 0x7f42e41036e0 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42ea5f9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41122e0 con 0x7f42e41082a0 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60242/0 (socket says 172.21.15.3:60242) 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 -- 172.21.15.3:0/3059641288 learned_addr learned my addr 172.21.15.3:0/3059641288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e8b96640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41082a0 0x7f42e419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e37fe640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e41036e0 0x7f42e4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:59.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 -- 172.21.15.3:0/3059641288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41082a0 msgr2=0x7f42e419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41082a0 0x7f42e419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 -- 172.21.15.3:0/3059641288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e41036e0 msgr2=0x7f42e4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e41036e0 0x7f42e4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.169+0000 7f42e3fff640 1 -- 172.21.15.3:0/3059641288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419d6f0 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42e3fff640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f42d400b4a0 tx=0x7f42d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4015020 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419d9e0 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4011d10 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400f2e0 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.170+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a52c0 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.171+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e406a850 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.172+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f42d4018020 con 0x7f42e4102d30 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.173+0000 7f42e17fa640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 0x7f42cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:26:59.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.173+0000 7f42e37fe640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 0x7f42cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:26:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.173+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f42d40ce8a0 con 0x7f42e4102d30 2024-09-12T02:26:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.173+0000 7f42e37fe640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 0x7f42cc0a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42e40fedc0 tx=0x7f42d8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:26:59.189 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.177+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d401f020 con 0x7f42e4102d30 2024-09-12T02:26:59.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.358+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42e410a2b0 con 0x7f42e4102d30 2024-09-12T02:26:59.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.360+0000 7f42e17fa640 1 -- 172.21.15.3:0/3059641288 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22542 (secure 0 0 0) 0x7f42d409b7f0 con 0x7f42e4102d30 2024-09-12T02:26:59.361 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-09-12T02:26:59.362 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":83,"fsid":"c595fa10-70ab-11ef-bcea-c7b262605968","created":"2024-09-12T02:08:39.465171+0000","modified":"2024-09-12T02:16:38.132385+0000","last_up_change":"2024-09-12T02:15:39.153049+0000","last_in_change":"2024-09-12T02:15:16.453774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:11:52.858011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:16:25.722169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:16:28.093977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d0aab17-f2a0-467b-9ab2-ddd8fc92d297","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6803","nonce":46072928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6805","nonce":46072928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6809","nonce":46072928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":46072928},{"type":"v1","addr":"172.21.15.3:6807","nonce":46072928}]},"public_addr":"172.21.15.3:6803/46072928","cluster_addr":"172.21.15.3:6805/46072928","heartbeat_back_addr":"172.21.15.3:6809/46072928","heartbeat_front_addr":"172.21.15.3:6807/46072928","state":["exists","up"]},{"osd":1,"uuid":"dcaeda7b-e940-4d6e-84b7-4b839bed3275","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6801","nonce":2606408940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6803","nonce":2606408940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6807","nonce":2606408940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":2606408940},{"type":"v1","addr":"172.21.15.26:6805","nonce":2606408940}]},"public_addr":"172.21.15.26:6801/2606408940","cluster_addr":"172.21.15.26:6803/2606408940","heartbeat_back_addr":"172.21.15.26:6807/2606408940","heartbeat_front_addr":"172.21.15.26:6805/2606408940","state":["exists","up"]},{"osd":2,"uuid":"6e76989d-dcc2-45a3-8f36-eb08674b7fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6801","nonce":3785909274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6803","nonce":3785909274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6807","nonce":3785909274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":3785909274},{"type":"v1","addr":"172.21.15.92:6805","nonce":3785909274}]},"public_addr":"172.21.15.92:6801/3785909274","cluster_addr":"172.21.15.92:6803/3785909274","heartbeat_back_addr":"172.21.15.92:6807/3785909274","heartbeat_front_addr":"172.21.15.92:6805/3785909274","state":["exists","up"]},{"osd":3,"uuid":"f79efe58-167d-445a-a4c8-47c4f46c1070","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6811","nonce":278455203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6813","nonce":278455203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6817","nonce":278455203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":278455203},{"type":"v1","addr":"172.21.15.3:6815","nonce":278455203}]},"public_addr":"172.21.15.3:6811/278455203","cluster_addr":"172.21.15.3:6813/278455203","heartbeat_back_addr":"172.21.15.3:6817/278455203","heartbeat_front_addr":"172.21.15.3:6815/278455203","state":["exists","up"]},{"osd":4,"uuid":"f3e2dea9-afc3-4c05-9c71-75758b7b311b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6809","nonce":525329948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6811","nonce":525329948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6815","nonce":525329948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":525329948},{"type":"v1","addr":"172.21.15.26:6813","nonce":525329948}]},"public_addr":"172.21.15.26:6809/525329948","cluster_addr":"172.21.15.26:6811/525329948","heartbeat_back_addr":"172.21.15.26:6815/525329948","heartbeat_front_addr":"172.21.15.26:6813/525329948","state":["exists","up"]},{"osd":5,"uuid":"6a9b28bd-0597-4710-b614-d90867985f7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6809","nonce":2340798273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6811","nonce":2340798273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6815","nonce":2340798273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2340798273},{"type":"v1","addr":"172.21.15.92:6813","nonce":2340798273}]},"public_addr":"172.21.15.92:6809/2340798273","cluster_addr":"172.21.15.92:6811/2340798273","heartbeat_back_addr":"172.21.15.92:6815/2340798273","heartbeat_front_addr":"172.21.15.92:6813/2340798273","state":["exists","up"]},{"osd":6,"uuid":"b1c4272b-d682-4423-924c-8500633b6e24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6819","nonce":342811658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6821","nonce":342811658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6825","nonce":342811658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":342811658},{"type":"v1","addr":"172.21.15.3:6823","nonce":342811658}]},"public_addr":"172.21.15.3:6819/342811658","cluster_addr":"172.21.15.3:6821/342811658","heartbeat_back_addr":"172.21.15.3:6825/342811658","heartbeat_front_addr":"172.21.15.3:6823/342811658","state":["exists","up"]},{"osd":7,"uuid":"3a14a8c6-dbf2-4a95-a670-d0b0dbf34dc0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6817","nonce":500611310}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6819","nonce":500611310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6823","nonce":500611310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":500611310},{"type":"v1","addr":"172.21.15.26:6821","nonce":500611310}]},"public_addr":"172.21.15.26:6817/500611310","cluster_addr":"172.21.15.26:6819/500611310","heartbeat_back_addr":"172.21.15.26:6823/500611310","heartbeat_front_addr":"172.21.15.26:6821/500611310","state":["exists","up"]},{"osd":8,"uuid":"e2c1241a-ec0f-4db8-b5d4-0e8895ddc469","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6817","nonce":1598499134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6819","nonce":1598499134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6823","nonce":1598499134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1598499134},{"type":"v1","addr":"172.21.15.92:6821","nonce":1598499134}]},"public_addr":"172.21.15.92:6817/1598499134","cluster_addr":"172.21.15.92:6819/1598499134","heartbeat_back_addr":"172.21.15.92:6823/1598499134","heartbeat_front_addr":"172.21.15.92:6821/1598499134","state":["exists","up"]},{"osd":9,"uuid":"7611642c-8229-45c0-b752-967b7260e2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6827","nonce":3263587345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6829","nonce":3263587345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6833","nonce":3263587345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":3263587345},{"type":"v1","addr":"172.21.15.3:6831","nonce":3263587345}]},"public_addr":"172.21.15.3:6827/3263587345","cluster_addr":"172.21.15.3:6829/3263587345","heartbeat_back_addr":"172.21.15.3:6833/3263587345","heartbeat_front_addr":"172.21.15.3:6831/3263587345","state":["exists","up"]},{"osd":10,"uuid":"1ab8f17b-02b0-441a-bddf-77679403a160","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6825","nonce":3810836866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6827","nonce":3810836866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6831","nonce":3810836866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":3810836866},{"type":"v1","addr":"172.21.15.26:6829","nonce":3810836866}]},"public_addr":"172.21.15.26:6825/3810836866","cluster_addr":"172.21.15.26:6827/3810836866","heartbeat_back_addr":"172.21.15.26:6831/3810836866","heartbeat_front_addr":"172.21.15.26:6829/3810836866","state":["exists","up"]},{"osd":11,"uuid":"84dbc141-f1cc-4f68-909e-53f67c8e5190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6825","nonce":3259822775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6827","nonce":3259822775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6831","nonce":3259822775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3259822775},{"type":"v1","addr":"172.21.15.92:6829","nonce":3259822775}]},"public_addr":"172.21.15.92:6825/3259822775","cluster_addr":"172.21.15.92:6827/3259822775","heartbeat_back_addr":"172.21.15.92:6831/3259822775","heartbeat_front_addr":"172.21.15.92:6829/3259822775","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:10:59.058831+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:23.330369+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:11:48.933015+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:15.182582+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:12:39.466798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:05.656569+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:30.861171+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:13:56.244755+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:22.215173+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:14:47.914154+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:11.842684+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:15:37.975229+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2921615271":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2551485583":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:0/2796707951":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/1673598194":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6801/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:6801/1539065682":"2024-09-13T02:09:36.346826+0000","172.21.15.3:6800/814335086":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/2395673224":"2024-09-13T02:09:12.115573+0000","172.21.15.3:0/3787188077":"2024-09-13T02:09:12.115573+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:26:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 msgr2=0x7f42cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 0x7f42cc0a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f42e40fedc0 tx=0x7f42d8009110 comp rx=0 tx=0).stop 2024-09-12T02:26:59.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 msgr2=0x7f42e4198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f42d400b4a0 tx=0x7f42d400b8a0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 shutdown_connections 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f42cc0a2b80 0x7f42cc0a5040 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e41082a0 0x7f42e419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42e41036e0 0x7f42e4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 --2- 172.21.15.3:0/3059641288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42e4102d30 0x7f42e4198740 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 >> 172.21.15.3:0/3059641288 conn(0x7f42e40fc7d0 msgr2=0x7f42e4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 shutdown_connections 2024-09-12T02:26:59.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:26:59.362+0000 7f42ea5f9640 1 -- 172.21.15.3:0/3059641288 wait complete. 2024-09-12T02:26:59.681 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:26:59 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3059641288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:59.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:26:59 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3059641288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:59.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:26:59 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3059641288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:26:59.986 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T02:27:00.244 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:27:00.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:00 smithi003 ceph-mon[31476]: pgmap v594: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- 172.21.15.3:0/1705949890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781082a0 msgr2=0x7f397810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 --2- 172.21.15.3:0/1705949890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781082a0 0x7f397810ec40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3970009cd0 tx=0x7f397002a910 comp rx=0 tx=0).stop 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- 172.21.15.3:0/1705949890 shutdown_connections 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 --2- 172.21.15.3:0/1705949890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781082a0 0x7f397810ec40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 --2- 172.21.15.3:0/1705949890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781036e0 0x7f3978107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 --2- 172.21.15.3:0/1705949890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3978102d30 0x7f3978103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- 172.21.15.3:0/1705949890 >> 172.21.15.3:0/1705949890 conn(0x7f39780fc7d0 msgr2=0x7f39780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- 172.21.15.3:0/1705949890 shutdown_connections 2024-09-12T02:27:00.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- 172.21.15.3:0/1705949890 wait complete. 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 Processor -- start 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.736+0000 7f397ed4f640 1 -- start start 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39781036e0 0x7f3978100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781082a0 0x7f3978101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112160 con 0x7f3978102d30 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f3978111fe0 con 0x7f39781082a0 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397ed4f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39781122e0 con 0x7f39781036e0 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60260/0 (socket says 172.21.15.3:60260) 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 -- 172.21.15.3:0/2083860049 learned_addr learned my addr 172.21.15.3:0/2083860049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:27:00.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397d2ec640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781082a0 0x7f3978101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f3977fff640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39781036e0 0x7f3978100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 -- 172.21.15.3:0/2083860049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39781036e0 msgr2=0x7f3978100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39781036e0 0x7f3978100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 -- 172.21.15.3:0/2083860049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781082a0 msgr2=0x7f3978101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781082a0 0x7f3978101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 -- 172.21.15.3:0/2083860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a6b80 con 0x7f3978102d30 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f397caeb640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f396800b4a0 tx=0x7f396800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968014070 con 0x7f3978102d30 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.737+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3968010af0 con 0x7f3978102d30 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.738+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a6e70 con 0x7f3978102d30 2024-09-12T02:27:00.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.738+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968017d10 con 0x7f3978102d30 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.738+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a72d0 con 0x7f3978102d30 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.739+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397806a850 con 0x7f3978102d30 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.740+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3968017070 con 0x7f3978102d30 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.740+0000 7f3975ffb640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 0x7f39640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.741+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8248+0+0 (secure 0 0 0) 0x7f39680ce480 con 0x7f3978102d30 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.741+0000 7f3977fff640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 0x7f39640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.741+0000 7f3977fff640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 0x7f39640a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3978102420 tx=0x7f3960008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:00.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.745+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f396809b370 con 0x7f3978102d30 2024-09-12T02:27:00.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:00 smithi026 ceph-mon[35496]: pgmap v594: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:27:00.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:00 smithi092 ceph-mon[36089]: pgmap v594: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T02:27:00.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.959+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f397810a2b0 con 0x7f3978102d30 2024-09-12T02:27:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.962+0000 7f3975ffb640 1 -- 172.21.15.3:0/2083860049 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7f396809ac70 con 0x7f3978102d30 2024-09-12T02:27:00.963 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 msgr2=0x7f39640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 0x7f39640a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3978102420 tx=0x7f3960008040 comp rx=0 tx=0).stop 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 msgr2=0x7f3978100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f396800b4a0 tx=0x7f396800b960 comp rx=0 tx=0).stop 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 shutdown_connections 2024-09-12T02:27:00.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f39640a2b80 0x7f39640a5040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39781082a0 0x7f3978101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39781036e0 0x7f3978100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 --2- 172.21.15.3:0/2083860049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3978102d30 0x7f3978100990 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f396800b4a0 tx=0x7f396800b960 comp rx=0 tx=0).stop 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.965+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 >> 172.21.15.3:0/2083860049 conn(0x7f39780fc7d0 msgr2=0x7f3978105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.966+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 shutdown_connections 2024-09-12T02:27:00.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:00.967+0000 7f397ed4f640 1 -- 172.21.15.3:0/2083860049 wait complete. 2024-09-12T02:27:01.278 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:00 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:27:00.960+0000 7f6df4851640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:27:01.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.279 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mds[72708]: -- 172.21.15.3:0/1961122892 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594293bd860 con 0x55942939b000 2024-09-12T02:27:01.279 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: starting mds.a at 2024-09-12T02:27:01.289 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.289 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.289 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[66249]: -- 172.21.15.92:0/6280292 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6f0b3d860 con 0x55e6f0b1b400 2024-09-12T02:27:01.289 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: starting mds.f at 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[66548]: -- 172.21.15.92:0/3524251359 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3b3a8f860 con 0x55d3b3a6d400 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: starting mds.c at 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[65945]: -- 172.21.15.92:0/3682086678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e43301cd00 con 0x55e432ff1000 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[65945]: -- 172.21.15.92:0/3682086678 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e433013860 con 0x55e432ff1000 2024-09-12T02:27:01.290 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: starting mds.l at 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[66856]: -- 172.21.15.92:0/1668593471 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b34b9d6680 con 0x55b34b9ab000 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mds[66856]: -- 172.21.15.92:0/1668593471 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b34b9cdc20 con 0x55b34b9ab000 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: starting mds.i at 2024-09-12T02:27:01.291 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:27:01.023+0000 7f3741511640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-09-12T02:27:01.298 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.298 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.298 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[65651]: -- 172.21.15.26:0/788278579 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad2002d860 con 0x55ad2000b000 2024-09-12T02:27:01.298 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: starting mds.b at 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[64724]: -- 172.21.15.26:0/3869923578 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bf5f90e680 con 0x55bf5f8e2800 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[64724]: -- 172.21.15.26:0/3869923578 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf5f905860 con 0x55bf5f8e2800 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: starting mds.k at 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.299 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[65338]: -- 172.21.15.26:0/1453952670 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55962d802680 con 0x55962d7d7400 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[65338]: -- 172.21.15.26:0/1453952670 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55962d7f9860 con 0x55962d7d7400 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: starting mds.h at 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e[65034]: ignoring --setuser ceph since I am not root 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e[65034]: ignoring --setgroup ceph since I am not root 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mds[65038]: -- 172.21.15.26:0/404059502 <== mon.1 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e3b103860 con 0x562e3b0e1000 2024-09-12T02:27:01.300 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e[65034]: starting mds.e at 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:27:01.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:27:01.719 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:01 smithi003 ceph-mon[31476]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T02:27:01.748 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T02:27:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:27:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:27:01.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:27:01.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:27:01.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:27:01.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:27:01.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:01 smithi026 ceph-mon[35496]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2083860049' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T02:27:01.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:01 smithi092 ceph-mon[36089]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T02:27:02.009 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 -- 172.21.15.3:0/472407549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c1049c0 msgr2=0x7f846c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 --2- 172.21.15.3:0/472407549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c1049c0 0x7f846c108a10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8460009900 tx=0x7f846002a410 comp rx=0 tx=0).stop 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 -- 172.21.15.3:0/472407549 shutdown_connections 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 --2- 172.21.15.3:0/472407549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c109140 0x7f846c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 --2- 172.21.15.3:0/472407549 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c1049c0 0x7f846c108a10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:02.581 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 --2- 172.21.15.3:0/472407549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c104010 0x7f846c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 -- 172.21.15.3:0/472407549 >> 172.21.15.3:0/472407549 conn(0x7f846c0ffa50 msgr2=0x7f846c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 -- 172.21.15.3:0/472407549 shutdown_connections 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.556+0000 7f8470acb640 1 -- 172.21.15.3:0/472407549 wait complete. 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 Processor -- start 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 -- start start 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c104010 0x7f846c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c1049c0 0x7f846c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f846c114140 con 0x7f846c109140 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f846c113fc0 con 0x7f846c1049c0 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.557+0000 7f8470acb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1142c0 con 0x7f846c104010 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:02.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60324/0 (socket says 172.21.15.3:60324) 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 -- 172.21.15.3:0/3251087584 learned_addr learned my addr 172.21.15.3:0/3251087584 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f8469d9b640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c1049c0 0x7f846c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846a59c640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c104010 0x7f846c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 -- 172.21.15.3:0/3251087584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c104010 msgr2=0x7f846c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c104010 0x7f846c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 -- 172.21.15.3:0/3251087584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c1049c0 msgr2=0x7f846c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c1049c0 0x7f846c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 -- 172.21.15.3:0/3251087584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c19f610 con 0x7f846c109140 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f846ad9d640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8464009fd0 tx=0x7f846400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464017070 con 0x7f846c109140 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8464003e50 con 0x7f846c109140 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c19f900 con 0x7f846c109140 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640106c0 con 0x7f846c109140 2024-09-12T02:27:02.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.558+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a71e0 con 0x7f846c109140 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.559+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c1043f0 con 0x7f846c109140 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.561+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8464003970 con 0x7f846c109140 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.561+0000 7f845b7fe640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 0x7f84540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.561+0000 7f846a59c640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 0x7f84540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.561+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8678+0+0 (secure 0 0 0) 0x7f84640ceee0 con 0x7f846c109140 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.562+0000 7f846a59c640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 0x7f84540a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f845c005d00 tx=0x7f845c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:02.584 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.566+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f846409b630 con 0x7f846c109140 2024-09-12T02:27:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: pgmap v595: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-12T02:27:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6834/1178814440,v1:172.21.15.92:6838/1178814440] up:boot 2024-09-12T02:27:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6835/1575934966,v1:172.21.15.92:6839/1575934966] up:boot 2024-09-12T02:27:02.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6832/1269286156,v1:172.21.15.92:6836/1269286156] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.92:6833/3035282598,v1:172.21.15.92:6837/3035282598] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6832/4051681762,v1:172.21.15.26:6836/4051681762] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6835/1689721226,v1:172.21.15.26:6839/1689721226] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6833/894744909,v1:172.21.15.26:6837/894744909] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.26:6834/2920621311,v1:172.21.15.26:6838/2920621311] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6838/4250300615,v1:172.21.15.3:6840/4250300615] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: mds.? [v2:172.21.15.3:6839/825660429,v1:172.21.15.3:6841/825660429] up:boot 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:27:02.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:27:02.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:02 smithi026 ceph-mon[35496]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: pgmap v595: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6834/1178814440,v1:172.21.15.92:6838/1178814440] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6835/1575934966,v1:172.21.15.92:6839/1575934966] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6832/1269286156,v1:172.21.15.92:6836/1269286156] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.92:6833/3035282598,v1:172.21.15.92:6837/3035282598] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6832/4051681762,v1:172.21.15.26:6836/4051681762] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6835/1689721226,v1:172.21.15.26:6839/1689721226] up:boot 2024-09-12T02:27:02.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6833/894744909,v1:172.21.15.26:6837/894744909] up:boot 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.26:6834/2920621311,v1:172.21.15.26:6838/2920621311] up:boot 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6838/4250300615,v1:172.21.15.3:6840/4250300615] up:boot 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: mds.? [v2:172.21.15.3:6839/825660429,v1:172.21.15.3:6841/825660429] up:boot 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:27:02.779 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:02 smithi003 ceph-mon[31476]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:27:02.780 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:02.779+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f846c10c380 con 0x7f846c109140 2024-09-12T02:27:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: pgmap v595: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-12T02:27:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6834/1178814440,v1:172.21.15.92:6838/1178814440] up:boot 2024-09-12T02:27:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6835/1575934966,v1:172.21.15.92:6839/1575934966] up:boot 2024-09-12T02:27:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6832/1269286156,v1:172.21.15.92:6836/1269286156] up:boot 2024-09-12T02:27:02.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.92:6833/3035282598,v1:172.21.15.92:6837/3035282598] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6832/4051681762,v1:172.21.15.26:6836/4051681762] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6835/1689721226,v1:172.21.15.26:6839/1689721226] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6833/894744909,v1:172.21.15.26:6837/894744909] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.26:6834/2920621311,v1:172.21.15.26:6838/2920621311] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6838/4250300615,v1:172.21.15.3:6840/4250300615] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: mds.? [v2:172.21.15.3:6839/825660429,v1:172.21.15.3:6841/825660429] up:boot 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:27:02.790 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:27:02.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:27:02.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:27:02.791 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:02 smithi092 ceph-mon[36089]: from='mgr.14150 172.21.15.3:0/382362548' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:27:03.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.425+0000 7f845b7fe640 1 -- 172.21.15.3:0/3251087584 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f846409f930 con 0x7f846c109140 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 msgr2=0x7f84540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 0x7f84540a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f845c005d00 tx=0x7f845c00b410 comp rx=0 tx=0).stop 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 msgr2=0x7f846c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8464009fd0 tx=0x7f846400b260 comp rx=0 tx=0).stop 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 shutdown_connections 2024-09-12T02:27:03.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f84540a2b80 0x7f84540a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f846c109140 0x7f846c19ef30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846c1049c0 0x7f846c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 --2- 172.21.15.3:0/3251087584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f846c104010 0x7f846c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 >> 172.21.15.3:0/3251087584 conn(0x7f846c0ffa50 msgr2=0x7f846c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 shutdown_connections 2024-09-12T02:27:03.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:03.428+0000 7f8470acb640 1 -- 172.21.15.3:0/3251087584 wait complete. 2024-09-12T02:27:03.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:03 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:27:03.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:03 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:27:03.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:03 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:27:04.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T02:27:04.430 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:27:04.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: pgmap v597: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:27:04.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:27:04.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:27:04.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: Cluster is now healthy 2024-09-12T02:27:04.685 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:27:04.686 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:04 smithi003 ceph-mon[31476]: fsmap 12 up:standby 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: pgmap v597: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: Cluster is now healthy 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:27:04.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:04 smithi026 ceph-mon[35496]: fsmap 12 up:standby 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: pgmap v597: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: Cluster is now healthy 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/3251087584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:27:04.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:04 smithi092 ceph-mon[36089]: fsmap 12 up:standby 2024-09-12T02:27:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 -- 172.21.15.3:0/100543230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff740102380 msgr2=0x7ff74010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 --2- 172.21.15.3:0/100543230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff740102380 0x7ff74010e9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff738009cd0 tx=0x7ff73802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 -- 172.21.15.3:0/100543230 shutdown_connections 2024-09-12T02:27:04.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 --2- 172.21.15.3:0/100543230 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff740102380 0x7ff74010e9c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 --2- 172.21.15.3:0/100543230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7401019e0 0x7ff740101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 --2- 172.21.15.3:0/100543230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 0x7ff740107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 -- 172.21.15.3:0/100543230 >> 172.21.15.3:0/100543230 conn(0x7ff7400fd700 msgr2=0x7ff7400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 -- 172.21.15.3:0/100543230 shutdown_connections 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.901+0000 7ff746d9d640 1 -- 172.21.15.3:0/100543230 wait complete. 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 Processor -- start 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 -- start start 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff740102380 0x7ff7401988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 0x7ff74019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111ef0 con 0x7ff7401019e0 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111d70 con 0x7ff740102380 2024-09-12T02:27:04.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff746d9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff740112070 con 0x7ff740107af0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff745d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff745d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60338/0 (socket says 172.21.15.3:60338) 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff745d9b640 1 -- 172.21.15.3:0/1928937315 learned_addr learned my addr 172.21.15.3:0/1928937315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff74559a640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff740102380 0x7ff7401988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.902+0000 7ff74659c640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 0x7ff74019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 -- 172.21.15.3:0/1928937315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 msgr2=0x7ff74019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 0x7ff74019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 -- 172.21.15.3:0/1928937315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff740102380 msgr2=0x7ff7401988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff740102380 0x7ff7401988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 -- 172.21.15.3:0/1928937315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74019d340 con 0x7ff7401019e0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff745d9b640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c970 tx=0x7ff73c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c007bb0 con 0x7ff7401019e0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff73c007d50 con 0x7ff7401019e0 2024-09-12T02:27:04.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74019d630 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00f450 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.903+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a4f10 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.904+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740101e40 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.905+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff73c005020 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.906+0000 7ff736ffd640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 0x7ff72c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.907+0000 7ff74559a640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 0x7ff72c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.907+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8678+0+0 (secure 0 0 0) 0x7ff73c0cee40 con 0x7ff7401019e0 2024-09-12T02:27:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.907+0000 7ff74559a640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 0x7ff72c0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff740106ab0 tx=0x7ff73000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:04.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:04.911+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff73c014030 con 0x7ff7401019e0 2024-09-12T02:27:05.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.091+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff740103270 con 0x7ff7401019e0 2024-09-12T02:27:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.427+0000 7ff736ffd640 1 -- 172.21.15.3:0/1928937315 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7ff73c09b590 con 0x7ff7401019e0 2024-09-12T02:27:05.429 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-09-12T02:27:05.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 msgr2=0x7ff72c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:05.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 0x7ff72c0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff740106ab0 tx=0x7ff73000b390 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 msgr2=0x7ff740198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c970 tx=0x7ff73c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 shutdown_connections 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7ff72c0a2c00 0x7ff72c0a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff740107af0 0x7ff74019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff740102380 0x7ff7401988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 --2- 172.21.15.3:0/1928937315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7401019e0 0x7ff740198390 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:05.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 >> 172.21.15.3:0/1928937315 conn(0x7ff7400fd700 msgr2=0x7ff7400fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 shutdown_connections 2024-09-12T02:27:05.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:05.431+0000 7ff746d9d640 1 -- 172.21.15.3:0/1928937315 wait complete. 2024-09-12T02:27:05.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:05 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:05.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:05 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:05.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:05 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:06.110 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid c595fa10-70ab-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T02:27:06.377 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:27:06.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:06 smithi003 ceph-mon[31476]: pgmap v598: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:27:06.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:06 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:06.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:06 smithi003 ceph-mon[31476]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:27:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:06 smithi026 ceph-mon[35496]: pgmap v598: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:27:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:06 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:06.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:06 smithi026 ceph-mon[35496]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:27:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:06 smithi092 ceph-mon[36089]: pgmap v598: 97 pgs: 97 active+clean; 81 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T02:27:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:06 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/1928937315' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:06.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:06 smithi092 ceph-mon[36089]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 -- 172.21.15.3:0/3182426640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34101420 msgr2=0x7f2e3410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/3182426640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34101420 0x7f2e3410d950 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c009cd0 tx=0x7f2e2c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 -- 172.21.15.3:0/3182426640 shutdown_connections 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/3182426640 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34101420 0x7f2e3410d950 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/3182426640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/3182426640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34106a80 0x7f2e34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 -- 172.21.15.3:0/3182426640 >> 172.21.15.3:0/3182426640 conn(0x7f2e340fc7a0 msgr2=0x7f2e340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 -- 172.21.15.3:0/3182426640 shutdown_connections 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.852+0000 7f2e3aabf640 1 -- 172.21.15.3:0/3182426640 wait complete. 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.853+0000 7f2e3aabf640 1 Processor -- start 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.853+0000 7f2e3aabf640 1 -- start start 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34101420 0x7f2e34197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34110e80 con 0x7f2e34106a80 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34110d00 con 0x7f2e34100a80 2024-09-12T02:27:06.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2e34111000 con 0x7f2e34101420 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60354/0 (socket says 172.21.15.3:60354) 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 -- 172.21.15.3:0/2888746022 learned_addr learned my addr 172.21.15.3:0/2888746022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3885b640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e33fff640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34101420 0x7f2e34197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 -- 172.21.15.3:0/2888746022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34101420 msgr2=0x7f2e34197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34101420 0x7f2e34197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 -- 172.21.15.3:0/2888746022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 msgr2=0x7f2e34197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 -- 172.21.15.3:0/2888746022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3419c2d0 con 0x7f2e34106a80 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3885b640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3905c640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005cc0 tx=0x7f2e2c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:06.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c004040 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3419c560 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e2c0046c0 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.854+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c046480 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.855+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341a3ea0 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.856+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e34100ee0 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.857+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2e2c0305a0 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.857+0000 7f2e31ffb640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 0x7f2e240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.858+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8182+0+0 (secure 0 0 0) 0x7f2e2c02e030 con 0x7f2e34106a80 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.858+0000 7f2e3885b640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 0x7f2e240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.858+0000 7f2e3885b640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 0x7f2e240a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005fd0 tx=0x7f2e280094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:27:06.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:06.862+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e2c04b060 con 0x7f2e34106a80 2024-09-12T02:27:07.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.041+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2e34109050 con 0x7f2e34106a80 2024-09-12T02:27:07.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.435+0000 7f2e31ffb640 1 -- 172.21.15.3:0/2888746022 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f2e2c0bd8b0 con 0x7f2e34106a80 2024-09-12T02:27:07.436 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-09-12T02:27:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 msgr2=0x7f2e240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 0x7f2e240a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e28005fd0 tx=0x7f2e280094d0 comp rx=0 tx=0).stop 2024-09-12T02:27:07.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 msgr2=0x7f2e3419bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005cc0 tx=0x7f2e2c003970 comp rx=0 tx=0).stop 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 shutdown_connections 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:6800/2546806763,v1:172.21.15.3:6801/2546806763] conn(0x7f2e240a2b80 0x7f2e240a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e34106a80 0x7f2e3419bbf0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e34101420 0x7f2e34197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 --2- 172.21.15.3:0/2888746022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e34100a80 0x7f2e34197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.439+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 >> 172.21.15.3:0/2888746022 conn(0x7f2e340fc7a0 msgr2=0x7f2e340fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.440+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 shutdown_connections 2024-09-12T02:27:07.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-09-12T02:27:07.440+0000 7f2e3aabf640 1 -- 172.21.15.3:0/2888746022 wait complete. 2024-09-12T02:27:07.683 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:07 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:07.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:07 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:07.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:07 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T02:27:08.256 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:27:08.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:27:08.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:27:08.395 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T02:27:08.395 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:27:08.411 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:27:08.431 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:27:08.456 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T02:27:08.456 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T02:27:08.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:27:08.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:08 smithi026 ceph-mon[35496]: pgmap v600: 65 pgs: 65 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:27:08.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:08 smithi026 ceph-mon[35496]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:08.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:08 smithi026 ceph-mon[35496]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:27:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 ceph-mon[31476]: pgmap v600: 65 pgs: 65 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:27:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 ceph-mon[31476]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 ceph-mon[31476]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:27:08.778 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 systemd[1]: Stopping Ceph mon.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:08 smithi092 ceph-mon[36089]: pgmap v600: 65 pgs: 65 active+clean; 577 KiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:27:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:08 smithi092 ceph-mon[36089]: from='client.? 172.21.15.3:0/2888746022' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T02:27:08.789 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:08 smithi092 ceph-mon[36089]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T02:27:09.256 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:27:08.788+0000 7f6dfa05c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:09.256 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:08 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a[31472]: 2024-09-12T02:27:08.788+0000 7f6dfa05c640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T02:27:09.256 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:09 smithi003 podman[100772]: 2024-09-12 02:27:09.013304189 +0000 UTC m=+0.346538987 container died 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T02:27:09.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:09 smithi003 podman[100772]: 2024-09-12 02:27:09.364412024 +0000 UTC m=+0.697646806 container cleanup 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T02:27:09.528 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:09 smithi003 bash[100772]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:27:09.853 INFO:journalctl@ceph.mon.a.smithi003.stdout:Sep 12 02:27:09 smithi003 podman[100785]: 2024-09-12 02:27:09.591209128 +0000 UTC m=+0.557748456 container remove 4fa36850ca4eaa1e167eee90eb50419a0a11a43508e3a69ca22a0dd7e12968be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:09.858 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T02:27:09.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:09.909 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T02:27:09.909 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T02:27:09.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.b 2024-09-12T02:27:10.234 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:09 smithi026 systemd[1]: Stopping Ceph mon.b for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:10 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-b[35492]: 2024-09-12T02:27:10.233+0000 7f7c404bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:10 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-b[35492]: 2024-09-12T02:27:10.233+0000 7f7c404bd640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:27:10.515 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:10 smithi026 podman[69407]: 2024-09-12 02:27:10.468346379 +0000 UTC m=+0.327841052 container died 0761ad51343b16b48c4c1c2936650ec79258d8c425aebfc9759395cbb969a1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T02:27:11.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:10 smithi026 podman[69407]: 2024-09-12 02:27:10.958480007 +0000 UTC m=+0.817974686 container cleanup 0761ad51343b16b48c4c1c2936650ec79258d8c425aebfc9759395cbb969a1e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:11.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Sep 12 02:27:10 smithi026 bash[69407]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-b 2024-09-12T02:27:11.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T02:27:11.589 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:11.589 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T02:27:11.589 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T02:27:11.589 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.c 2024-09-12T02:27:11.992 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:11 smithi092 systemd[1]: Stopping Ceph mon.c for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:12.246 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:11 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-c[36085]: 2024-09-12T02:27:11.991+0000 7f8af7117640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:12.246 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:11 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-c[36085]: 2024-09-12T02:27:11.991+0000 7f8af7117640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T02:27:12.539 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:12 smithi092 podman[70421]: 2024-09-12 02:27:12.246212457 +0000 UTC m=+0.386700363 container died 99eca5e2f7c86272fdd92e1ab08e14d17b42e9d9595d587ab660c3c478ec7bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:12.927 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:12 smithi092 podman[70421]: 2024-09-12 02:27:12.675346491 +0000 UTC m=+0.815834409 container cleanup 99eca5e2f7c86272fdd92e1ab08e14d17b42e9d9595d587ab660c3c478ec7bd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T02:27:12.927 INFO:journalctl@ceph.mon.c.smithi092.stdout:Sep 12 02:27:12 smithi092 bash[70421]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mon-c 2024-09-12T02:27:13.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T02:27:13.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:13.188 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T02:27:13.188 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T02:27:13.188 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:27:13.528 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:27:13 smithi003 systemd[1]: Stopping Ceph mgr.x for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:13.996 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:27:13 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x[31701]: 2024-09-12T02:27:13.659+0000 7fb678c9d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:13.996 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:27:13 smithi003 podman[100895]: 2024-09-12 02:27:13.69276931 +0000 UTC m=+0.304652583 container died 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T02:27:14.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:27:14 smithi003 podman[100895]: 2024-09-12 02:27:14.120721282 +0000 UTC m=+0.732604542 container cleanup 91cdfa5dea8ca11f261c75c3cd083f370d3c59fbdf4ac72a4b1de30a0551f974 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:14.278 INFO:journalctl@ceph.mgr.x.smithi003.stdout:Sep 12 02:27:14 smithi003 bash[100895]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-x 2024-09-12T02:27:14.586 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T02:27:14.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:14.624 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T02:27:14.625 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T02:27:14.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.y 2024-09-12T02:27:14.963 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:14 smithi026 systemd[1]: Stopping Ceph mgr.y for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:15.265 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:14 smithi026 podman[69530]: 2024-09-12 02:27:14.963275015 +0000 UTC m=+0.138888289 container died 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:15.624 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:15 smithi026 podman[69530]: 2024-09-12 02:27:15.397756135 +0000 UTC m=+0.573369447 container cleanup 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T02:27:15.624 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:15 smithi026 bash[69530]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y 2024-09-12T02:27:15.880 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:15 smithi026 podman[69542]: 2024-09-12 02:27:15.624466366 +0000 UTC m=+0.651889990 container remove 5ed8fdbe058e9e1807b7df6b51a768cb4da252b05e7319f9a348a6069e807c99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T02:27:15.880 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Sep 12 02:27:15 smithi026 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-12T02:27:15.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T02:27:15.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:15.919 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T02:27:15.919 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T02:27:15.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z 2024-09-12T02:27:16.277 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:15 smithi092 systemd[1]: Stopping Ceph mgr.z for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:16.539 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:16 smithi092 podman[70545]: 2024-09-12 02:27:16.27741633 +0000 UTC m=+0.159650400 container died 9f5bec7348959b58160330c000cfac8807de8d82da0bd40c34d2c3c66b2de4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:27:16.933 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:16 smithi092 podman[70545]: 2024-09-12 02:27:16.706217415 +0000 UTC m=+0.588451480 container cleanup 9f5bec7348959b58160330c000cfac8807de8d82da0bd40c34d2c3c66b2de4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T02:27:16.933 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:16 smithi092 bash[70545]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z 2024-09-12T02:27:17.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T02:27:17.201 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:16 smithi092 podman[70557]: 2024-09-12 02:27:16.932980249 +0000 UTC m=+0.645086003 container remove 9f5bec7348959b58160330c000cfac8807de8d82da0bd40c34d2c3c66b2de4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T02:27:17.201 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:16 smithi092 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-12T02:27:17.201 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:17 smithi092 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-12T02:27:17.201 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:17 smithi092 systemd[1]: Stopped Ceph mgr.z for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:27:17.201 INFO:journalctl@ceph.mgr.z.smithi092.stdout:Sep 12 02:27:17 smithi092 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mgr.z.service: Consumed 6.450s CPU time. 2024-09-12T02:27:17.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:17.214 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T02:27:17.214 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T02:27:17.214 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.0 2024-09-12T02:27:17.528 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:17 smithi003 systemd[1]: Stopping Ceph osd.0 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:18.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:27:17.533+0000 7f87fd754640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:18.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:27:17.533+0000 7f87fd754640 -1 osd.0 86 *** Got signal Terminated *** 2024-09-12T02:27:18.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:17 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0[45902]: 2024-09-12T02:27:17.533+0000 7f87fd754640 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:27:23.364 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:23 smithi003 podman[101015]: 2024-09-12 02:27:23.067373399 +0000 UTC m=+5.646482552 container died 5b50a2470d97a4c3f23c5cd8f6d0d9b7b17cf243a306d685345ed3bf235adae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-12T02:27:23.744 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:23 smithi003 podman[101015]: 2024-09-12 02:27:23.459175843 +0000 UTC m=+6.038285035 container cleanup 5b50a2470d97a4c3f23c5cd8f6d0d9b7b17cf243a306d685345ed3bf235adae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T02:27:23.744 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:23 smithi003 bash[101015]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0 2024-09-12T02:27:24.028 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:23 smithi003 podman[101032]: 2024-09-12 02:27:23.744292629 +0000 UTC m=+0.672843327 container remove 5b50a2470d97a4c3f23c5cd8f6d0d9b7b17cf243a306d685345ed3bf235adae2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T02:27:24.355 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:23.983803951 +0000 UTC m=+0.016254389 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:27:24.355 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:24.128949864 +0000 UTC m=+0.161400292 container create cb04994c20a2804d43c31a4c8b7881887e56e57e31e626968895b7eba3b5eb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T02:27:24.778 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:24.490144891 +0000 UTC m=+0.522595321 container init cb04994c20a2804d43c31a4c8b7881887e56e57e31e626968895b7eba3b5eb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:27:24.778 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:24.493542834 +0000 UTC m=+0.525993263 container start cb04994c20a2804d43c31a4c8b7881887e56e57e31e626968895b7eba3b5eb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:27:24.778 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:24.573781624 +0000 UTC m=+0.606232065 container attach cb04994c20a2804d43c31a4c8b7881887e56e57e31e626968895b7eba3b5eb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:27:25.177 INFO:journalctl@ceph.osd.0.smithi003.stdout:Sep 12 02:27:24 smithi003 podman[101121]: 2024-09-12 02:27:24.868658956 +0000 UTC m=+0.901109398 container died cb04994c20a2804d43c31a4c8b7881887e56e57e31e626968895b7eba3b5eb26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T02:27:25.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T02:27:25.544 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:25.544 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T02:27:25.544 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T02:27:25.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.1 2024-09-12T02:27:25.836 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:25 smithi026 systemd[1]: Stopping Ceph osd.1 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:26.265 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:25 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:27:25.834+0000 7f81f7b46640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:26.265 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:25 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:27:25.834+0000 7f81f7b46640 -1 osd.1 86 *** Got signal Terminated *** 2024-09-12T02:27:26.265 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:25 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1[39939]: 2024-09-12T02:27:25.834+0000 7f81f7b46640 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:27:31.753 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:31 smithi026 podman[69668]: 2024-09-12 02:27:31.438719511 +0000 UTC m=+5.698415271 container died ef79b7e25ce19584dfa2a987ba9dbf021fbd3f32a7236118a1b04c057d910312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:32.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:31 smithi026 podman[69668]: 2024-09-12 02:27:31.96917026 +0000 UTC m=+6.228866019 container cleanup ef79b7e25ce19584dfa2a987ba9dbf021fbd3f32a7236118a1b04c057d910312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T02:27:32.015 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:31 smithi026 bash[69668]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1 2024-09-12T02:27:32.515 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:32 smithi026 podman[69684]: 2024-09-12 02:27:32.221069669 +0000 UTC m=+0.774281866 container remove ef79b7e25ce19584dfa2a987ba9dbf021fbd3f32a7236118a1b04c057d910312 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:27:32.933 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:32 smithi026 podman[69769]: 2024-09-12 02:27:32.549118562 +0000 UTC m=+0.015680392 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:27:32.933 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:32 smithi026 podman[69769]: 2024-09-12 02:27:32.663872835 +0000 UTC m=+0.130434693 container create 2919e72807bf5fd57659c421f93009be5411664acc6486877c18c5b585cee124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T02:27:33.225 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:32 smithi026 podman[69769]: 2024-09-12 02:27:32.933424972 +0000 UTC m=+0.399986797 container init 2919e72807bf5fd57659c421f93009be5411664acc6486877c18c5b585cee124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T02:27:33.225 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:32 smithi026 podman[69769]: 2024-09-12 02:27:32.937165043 +0000 UTC m=+0.403726859 container start 2919e72807bf5fd57659c421f93009be5411664acc6486877c18c5b585cee124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T02:27:33.225 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:33 smithi026 podman[69769]: 2024-09-12 02:27:33.050112456 +0000 UTC m=+0.516674290 container attach 2919e72807bf5fd57659c421f93009be5411664acc6486877c18c5b585cee124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T02:27:33.515 INFO:journalctl@ceph.osd.1.smithi026.stdout:Sep 12 02:27:33 smithi026 podman[69769]: 2024-09-12 02:27:33.225904467 +0000 UTC m=+0.692466287 container died 2919e72807bf5fd57659c421f93009be5411664acc6486877c18c5b585cee124 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T02:27:33.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T02:27:33.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:33.919 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T02:27:33.919 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T02:27:33.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.2 2024-09-12T02:27:34.252 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:33 smithi092 systemd[1]: Stopping Ceph osd.2 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:34.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:27:34.251+0000 7f368a085640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:34.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:27:34.251+0000 7f368a085640 -1 osd.2 86 *** Got signal Terminated *** 2024-09-12T02:27:34.539 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2[41028]: 2024-09-12T02:27:34.251+0000 7f368a085640 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:27:40.120 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:39 smithi092 podman[70683]: 2024-09-12 02:27:39.812336067 +0000 UTC m=+5.694694600 container died 1f1e6ec49af4322bc3392c779965ec049e8f95e4dfbf8198953fe0a7de5fde22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:27:40.512 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:40 smithi092 podman[70683]: 2024-09-12 02:27:40.226666391 +0000 UTC m=+6.109024979 container cleanup 1f1e6ec49af4322bc3392c779965ec049e8f95e4dfbf8198953fe0a7de5fde22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T02:27:40.512 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:40 smithi092 bash[70683]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2 2024-09-12T02:27:40.789 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:40 smithi092 podman[70697]: 2024-09-12 02:27:40.511690611 +0000 UTC m=+0.687808404 container remove 1f1e6ec49af4322bc3392c779965ec049e8f95e4dfbf8198953fe0a7de5fde22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:27:41.090 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:40 smithi092 podman[70789]: 2024-09-12 02:27:40.726828028 +0000 UTC m=+0.015515917 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:27:41.090 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:40 smithi092 podman[70789]: 2024-09-12 02:27:40.862738977 +0000 UTC m=+0.151426921 container create 92ff0029805ba57462e711fceed6b3f44e5f36b854765478acb129f6797b1b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:27:41.379 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:41 smithi092 podman[70789]: 2024-09-12 02:27:41.090545878 +0000 UTC m=+0.379233756 container init 92ff0029805ba57462e711fceed6b3f44e5f36b854765478acb129f6797b1b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:41.379 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:41 smithi092 podman[70789]: 2024-09-12 02:27:41.094264074 +0000 UTC m=+0.382951953 container start 92ff0029805ba57462e711fceed6b3f44e5f36b854765478acb129f6797b1b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T02:27:41.379 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:41 smithi092 podman[70789]: 2024-09-12 02:27:41.215536845 +0000 UTC m=+0.504224723 container attach 92ff0029805ba57462e711fceed6b3f44e5f36b854765478acb129f6797b1b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:41.686 INFO:journalctl@ceph.osd.2.smithi092.stdout:Sep 12 02:27:41 smithi092 podman[70789]: 2024-09-12 02:27:41.37915827 +0000 UTC m=+0.667846158 container died 92ff0029805ba57462e711fceed6b3f44e5f36b854765478acb129f6797b1b07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:27:41.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T02:27:42.026 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:42.026 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T02:27:42.027 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T02:27:42.027 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.3 2024-09-12T02:27:42.356 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:42 smithi003 systemd[1]: Stopping Ceph osd.3 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:42.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:42 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:27:42.355+0000 7fe4fd47a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:42.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:42 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:27:42.355+0000 7fe4fd47a640 -1 osd.3 86 *** Got signal Terminated *** 2024-09-12T02:27:42.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:42 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3[52350]: 2024-09-12T02:27:42.355+0000 7fe4fd47a640 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:27:48.273 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:47 smithi003 podman[101235]: 2024-09-12 02:27:47.999601902 +0000 UTC m=+5.756311959 container died 92c14d2f6ea912457bd8c3a3e5df142e7cb029fa7cf7f81d741a1a4c52490e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:48.528 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:48 smithi003 podman[101235]: 2024-09-12 02:27:48.463673298 +0000 UTC m=+6.220383358 container cleanup 92c14d2f6ea912457bd8c3a3e5df142e7cb029fa7cf7f81d741a1a4c52490e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:48.528 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:48 smithi003 bash[101235]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3 2024-09-12T02:27:49.028 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:48 smithi003 podman[101250]: 2024-09-12 02:27:48.756422652 +0000 UTC m=+0.756372723 container remove 92c14d2f6ea912457bd8c3a3e5df142e7cb029fa7cf7f81d741a1a4c52490e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:27:49.351 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101340]: 2024-09-12 02:27:48.973356837 +0000 UTC m=+0.016059916 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:27:49.351 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101340]: 2024-09-12 02:27:49.107371198 +0000 UTC m=+0.150074262 container create 29f2ccad9755c34e3b81d63a1502647f426b982cb093bb2b786b9c869a9b03a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T02:27:49.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101340]: 2024-09-12 02:27:49.351958175 +0000 UTC m=+0.394661240 container init 29f2ccad9755c34e3b81d63a1502647f426b982cb093bb2b786b9c869a9b03a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:27:49.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101340]: 2024-09-12 02:27:49.355740908 +0000 UTC m=+0.398443976 container start 29f2ccad9755c34e3b81d63a1502647f426b982cb093bb2b786b9c869a9b03a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T02:27:49.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101340]: 2024-09-12 02:27:49.452006964 +0000 UTC m=+0.494710039 container attach 29f2ccad9755c34e3b81d63a1502647f426b982cb093bb2b786b9c869a9b03a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T02:27:49.778 INFO:journalctl@ceph.osd.3.smithi003.stdout:Sep 12 02:27:49 smithi003 podman[101363]: 2024-09-12 02:27:49.569770221 +0000 UTC m=+0.016722973 container died 29f2ccad9755c34e3b81d63a1502647f426b982cb093bb2b786b9c869a9b03a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T02:27:50.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T02:27:50.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:50.212 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T02:27:50.212 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T02:27:50.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.4 2024-09-12T02:27:50.515 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:50 smithi026 systemd[1]: Stopping Ceph osd.4 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:51.015 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:50 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:27:50.523+0000 7f8a049d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:51.015 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:50 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:27:50.523+0000 7f8a049d1640 -1 osd.4 86 *** Got signal Terminated *** 2024-09-12T02:27:51.015 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:50 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4[46832]: 2024-09-12T02:27:50.523+0000 7f8a049d1640 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:27:56.332 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:56 smithi026 podman[69871]: 2024-09-12 02:27:56.029924406 +0000 UTC m=+5.620873809 container died c61d70f1669e55035f09249b8ea123998d8c880d3b374d9f99c0856873989f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T02:27:56.674 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:56 smithi026 podman[69871]: 2024-09-12 02:27:56.423002218 +0000 UTC m=+6.013951619 container cleanup c61d70f1669e55035f09249b8ea123998d8c880d3b374d9f99c0856873989f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T02:27:56.674 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:56 smithi026 bash[69871]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4 2024-09-12T02:27:56.988 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:56 smithi026 podman[69886]: 2024-09-12 02:27:56.674416117 +0000 UTC m=+0.645733747 container remove c61d70f1669e55035f09249b8ea123998d8c880d3b374d9f99c0856873989f47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T02:27:57.265 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:56 smithi026 podman[69973]: 2024-09-12 02:27:56.890985525 +0000 UTC m=+0.015939875 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:27:57.265 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:57 smithi026 podman[69973]: 2024-09-12 02:27:57.025592352 +0000 UTC m=+0.150546692 container create be95b470dc6f9190192a89fee5f60ecbcff8d31b1e56e45157e5e86f93143c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T02:27:57.765 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:57 smithi026 podman[69973]: 2024-09-12 02:27:57.292043485 +0000 UTC m=+0.416997838 container init be95b470dc6f9190192a89fee5f60ecbcff8d31b1e56e45157e5e86f93143c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:27:57.765 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:57 smithi026 podman[69973]: 2024-09-12 02:27:57.295954295 +0000 UTC m=+0.420908642 container start be95b470dc6f9190192a89fee5f60ecbcff8d31b1e56e45157e5e86f93143c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T02:27:57.765 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:57 smithi026 podman[69973]: 2024-09-12 02:27:57.386725203 +0000 UTC m=+0.511679557 container attach be95b470dc6f9190192a89fee5f60ecbcff8d31b1e56e45157e5e86f93143c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T02:27:57.766 INFO:journalctl@ceph.osd.4.smithi026.stdout:Sep 12 02:27:57 smithi026 podman[69973]: 2024-09-12 02:27:57.488477692 +0000 UTC m=+0.613432033 container died be95b470dc6f9190192a89fee5f60ecbcff8d31b1e56e45157e5e86f93143c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:27:58.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T02:27:58.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:27:58.276 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T02:27:58.276 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T02:27:58.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.5 2024-09-12T02:27:58.597 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:27:58 smithi092 systemd[1]: Stopping Ceph osd.5 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:27:59.039 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:27:58 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:27:58.596+0000 7f686c885640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:27:59.039 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:27:58 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:27:58.596+0000 7f686c885640 -1 osd.5 86 *** Got signal Terminated *** 2024-09-12T02:27:59.039 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:27:58 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5[47923]: 2024-09-12T02:27:58.596+0000 7f686c885640 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:04.289 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:04 smithi092 podman[70915]: 2024-09-12 02:28:04.009453688 +0000 UTC m=+5.533809934 container died dbd339c33e871282a6b58a7ee6d79007c5ec77c5719b4773b377444683341b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T02:28:04.723 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:04 smithi092 podman[70915]: 2024-09-12 02:28:04.438561866 +0000 UTC m=+5.962918092 container cleanup dbd339c33e871282a6b58a7ee6d79007c5ec77c5719b4773b377444683341b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:04.723 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:04 smithi092 bash[70915]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5 2024-09-12T02:28:05.039 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:04 smithi092 podman[70930]: 2024-09-12 02:28:04.723706082 +0000 UTC m=+0.707284237 container remove dbd339c33e871282a6b58a7ee6d79007c5ec77c5719b4773b377444683341b0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T02:28:05.469 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.113454369 +0000 UTC m=+0.015771777 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:05.469 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.233003958 +0000 UTC m=+0.135321358 container create cc26be179ebc096e19b83dc84289cf029883d082222d37685f6fc058e7732b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T02:28:05.469 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.469290924 +0000 UTC m=+0.371608336 container init cc26be179ebc096e19b83dc84289cf029883d082222d37685f6fc058e7732b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:28:05.789 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.47304231 +0000 UTC m=+0.375359713 container start cc26be179ebc096e19b83dc84289cf029883d082222d37685f6fc058e7732b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:05.789 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.577423608 +0000 UTC m=+0.479741025 container attach cc26be179ebc096e19b83dc84289cf029883d082222d37685f6fc058e7732b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T02:28:05.789 INFO:journalctl@ceph.osd.5.smithi092.stdout:Sep 12 02:28:05 smithi092 podman[71022]: 2024-09-12 02:28:05.661058915 +0000 UTC m=+0.563376323 container died cc26be179ebc096e19b83dc84289cf029883d082222d37685f6fc058e7732b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:28:06.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T02:28:06.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:06.445 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T02:28:06.446 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T02:28:06.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.6 2024-09-12T02:28:06.734 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:06 smithi003 systemd[1]: Stopping Ceph osd.6 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:07.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:06 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:28:06.733+0000 7f8be0f44640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:07.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:06 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:28:06.733+0000 7f8be0f44640 -1 osd.6 86 *** Got signal Terminated *** 2024-09-12T02:28:07.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:06 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6[58980]: 2024-09-12T02:28:06.733+0000 7f8be0f44640 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:12.528 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:12 smithi003 podman[101458]: 2024-09-12 02:28:12.316430535 +0000 UTC m=+5.663019917 container died 2b7b5e3673a9feaa3db45c5b763db0579c01c6ca6876455349f3f75d1bdcff99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:13.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:12 smithi003 podman[101458]: 2024-09-12 02:28:12.766836654 +0000 UTC m=+6.113426036 container cleanup 2b7b5e3673a9feaa3db45c5b763db0579c01c6ca6876455349f3f75d1bdcff99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T02:28:13.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:12 smithi003 bash[101458]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6 2024-09-12T02:28:13.427 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101474]: 2024-09-12 02:28:13.118572652 +0000 UTC m=+0.799786312 container remove 2b7b5e3673a9feaa3db45c5b763db0579c01c6ca6876455349f3f75d1bdcff99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T02:28:13.730 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.427812136 +0000 UTC m=+0.106494976 container create 5956d84ff15d1c8c71b5e773200f076183417f5fc0586d8c501f48fa46e43d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T02:28:13.730 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.337462981 +0000 UTC m=+0.016145785 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:13.730 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.638707309 +0000 UTC m=+0.317390115 container init 5956d84ff15d1c8c71b5e773200f076183417f5fc0586d8c501f48fa46e43d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T02:28:13.730 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.642050177 +0000 UTC m=+0.320732976 container start 5956d84ff15d1c8c71b5e773200f076183417f5fc0586d8c501f48fa46e43d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:13.730 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.730366748 +0000 UTC m=+0.409049587 container attach 5956d84ff15d1c8c71b5e773200f076183417f5fc0586d8c501f48fa46e43d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T02:28:14.028 INFO:journalctl@ceph.osd.6.smithi003.stdout:Sep 12 02:28:13 smithi003 podman[101562]: 2024-09-12 02:28:13.838571801 +0000 UTC m=+0.517254596 container died 5956d84ff15d1c8c71b5e773200f076183417f5fc0586d8c501f48fa46e43d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-12T02:28:14.417 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T02:28:14.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:14.465 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T02:28:14.465 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T02:28:14.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.7 2024-09-12T02:28:14.765 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:14 smithi026 systemd[1]: Stopping Ceph osd.7 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:15.265 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:14 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:28:14.767+0000 7febcb3d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:15.265 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:14 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:28:14.767+0000 7febcb3d8640 -1 osd.7 86 *** Got signal Terminated *** 2024-09-12T02:28:15.265 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:14 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7[53515]: 2024-09-12T02:28:14.767+0000 7febcb3d8640 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:20.515 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:20 smithi026 podman[70087]: 2024-09-12 02:28:20.204971953 +0000 UTC m=+5.539899393 container died f001592fc62ad51c5dea45aac2e73960a3cba514b232c11a555adc9640956fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:20.791 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:20 smithi026 podman[70087]: 2024-09-12 02:28:20.530031924 +0000 UTC m=+5.864959363 container cleanup f001592fc62ad51c5dea45aac2e73960a3cba514b232c11a555adc9640956fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T02:28:20.791 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:20 smithi026 bash[70087]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7 2024-09-12T02:28:21.195 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:20 smithi026 podman[70103]: 2024-09-12 02:28:20.7913161 +0000 UTC m=+0.580593716 container remove f001592fc62ad51c5dea45aac2e73960a3cba514b232c11a555adc9640956fbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T02:28:21.471 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.097042078 +0000 UTC m=+0.016186024 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:21.471 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.211214226 +0000 UTC m=+0.130358193 container create 8fd49537dcb0baaa1747a0195ab91bbd575df994a5b05c25edc3be93d0de5e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:28:21.471 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.472004243 +0000 UTC m=+0.391148171 container init 8fd49537dcb0baaa1747a0195ab91bbd575df994a5b05c25edc3be93d0de5e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T02:28:21.765 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.475738047 +0000 UTC m=+0.394881972 container start 8fd49537dcb0baaa1747a0195ab91bbd575df994a5b05c25edc3be93d0de5e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:28:21.765 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.580401754 +0000 UTC m=+0.499545697 container attach 8fd49537dcb0baaa1747a0195ab91bbd575df994a5b05c25edc3be93d0de5e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:21.765 INFO:journalctl@ceph.osd.7.smithi026.stdout:Sep 12 02:28:21 smithi026 podman[70192]: 2024-09-12 02:28:21.675150792 +0000 UTC m=+0.594294731 container died 8fd49537dcb0baaa1747a0195ab91bbd575df994a5b05c25edc3be93d0de5e59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T02:28:22.411 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T02:28:22.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:22.456 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T02:28:22.456 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T02:28:22.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.8 2024-09-12T02:28:22.766 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:22 smithi092 systemd[1]: Stopping Ceph osd.8 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:23.039 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:22 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:28:22.765+0000 7f98acad5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:23.039 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:22 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:28:22.765+0000 7f98acad5640 -1 osd.8 86 *** Got signal Terminated *** 2024-09-12T02:28:23.039 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:22 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8[54572]: 2024-09-12T02:28:22.765+0000 7f98acad5640 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:24.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:23 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:23.862+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:25.015 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:24 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:24.520+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:25.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:24 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:24.872+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:25.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:25 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:25.479+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:26.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:25 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:25.871+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:26.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:26 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:26.253+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:26.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:26 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:26.477+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:27.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:26 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:26.899+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:27.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:27 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:27.459+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:27.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:27 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:27.282+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:28.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:27 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:27.886+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:28.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:28 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:28.246+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:28.757 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:28 smithi092 podman[71124]: 2024-09-12 02:28:28.45708907 +0000 UTC m=+5.812058313 container died 1c69ee7dd72bbf78321a262c0e52db6eeecb311fbdbaac5c3502de09a4e10948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:28.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:28 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:28.440+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:29.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:28 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:28.919+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:29.039 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:28 smithi092 podman[71124]: 2024-09-12 02:28:28.865371412 +0000 UTC m=+6.220340655 container cleanup 1c69ee7dd72bbf78321a262c0e52db6eeecb311fbdbaac5c3502de09a4e10948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:29.040 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:28 smithi092 bash[71124]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8 2024-09-12T02:28:29.439 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71139]: 2024-09-12 02:28:29.13412827 +0000 UTC m=+0.678149556 container remove 1c69ee7dd72bbf78321a262c0e52db6eeecb311fbdbaac5c3502de09a4e10948 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:29.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:29 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:29.230+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:29.704 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.341787835 +0000 UTC m=+0.015285134 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:29.704 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.476588887 +0000 UTC m=+0.150086186 container create 8d39e9eae08fc74af65725521705a6378201ec5e1c7fa6044e2880e4f365fa05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T02:28:29.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:29 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:29.467+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:30.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:29 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:29.943+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:30.039 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.704189806 +0000 UTC m=+0.377687109 container init 8d39e9eae08fc74af65725521705a6378201ec5e1c7fa6044e2880e4f365fa05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T02:28:30.040 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.70786702 +0000 UTC m=+0.381364314 container start 8d39e9eae08fc74af65725521705a6378201ec5e1c7fa6044e2880e4f365fa05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T02:28:30.040 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.795716003 +0000 UTC m=+0.469213297 container attach 8d39e9eae08fc74af65725521705a6378201ec5e1c7fa6044e2880e4f365fa05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:30.040 INFO:journalctl@ceph.osd.8.smithi092.stdout:Sep 12 02:28:29 smithi092 podman[71227]: 2024-09-12 02:28:29.901875344 +0000 UTC m=+0.575372638 container died 8d39e9eae08fc74af65725521705a6378201ec5e1c7fa6044e2880e4f365fa05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T02:28:30.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:30.270+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:30.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T02:28:30.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:30.652 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T02:28:30.653 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T02:28:30.653 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.9 2024-09-12T02:28:30.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:30 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:30.511+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:30.914 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:30 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:30.914+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:30.986 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:30 smithi003 systemd[1]: Stopping Ceph osd.9 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:30.986 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:30.984+0000 7eff4e7ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:30.986 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:30.984+0000 7eff4e7ae640 -1 osd.9 86 *** Got signal Terminated *** 2024-09-12T02:28:30.986 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:30 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:30.984+0000 7eff4e7ae640 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:31.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:31 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:31.480+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:31.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:31 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:31.316+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:32.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:31 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:31.937+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:32.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:32.313+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:32.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:32 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:32.313+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:30.445797+0000 front 2024-09-12T02:27:30.445467+0000 (oldest deadline 2024-09-12T02:28:32.145437+0000) 2024-09-12T02:28:32.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:32 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:32.430+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:32.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:32 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:32.430+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:33.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:32 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:32.911+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:33.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:33 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:33.383+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:33.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:33 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:33.383+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:33.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:33 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:33.339+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:33.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:33 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:33.339+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:30.445797+0000 front 2024-09-12T02:27:30.445467+0000 (oldest deadline 2024-09-12T02:28:32.145437+0000) 2024-09-12T02:28:34.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:33 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:33.906+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:34.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:34 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:34.417+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:34.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:34 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:34.417+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:34.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:34.382+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:34.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:34 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:34.382+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:30.445797+0000 front 2024-09-12T02:27:30.445467+0000 (oldest deadline 2024-09-12T02:28:32.145437+0000) 2024-09-12T02:28:35.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:34.894+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:35.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:35 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:35.436+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:35.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:35 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:35.436+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:35.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:35.371+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:21.143673+0000 front 2024-09-12T02:27:21.143613+0000 (oldest deadline 2024-09-12T02:28:25.843774+0000) 2024-09-12T02:28:35.778 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:35 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9[65637]: 2024-09-12T02:28:35.371+0000 7eff46dbc640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:30.445797+0000 front 2024-09-12T02:27:30.445467+0000 (oldest deadline 2024-09-12T02:28:32.145437+0000) 2024-09-12T02:28:36.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:35 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:35.906+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:36.721 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:36.417+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:36.722 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:36 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:36.417+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:36.745 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:36 smithi003 podman[101665]: 2024-09-12 02:28:36.456156891 +0000 UTC m=+5.587006213 container died 12d4436d5b01b2328118320746c028fafb50eb62cd3635f5caa475a818aa17d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T02:28:37.028 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:36 smithi003 podman[101665]: 2024-09-12 02:28:36.902412685 +0000 UTC m=+6.033262006 container cleanup 12d4436d5b01b2328118320746c028fafb50eb62cd3635f5caa475a818aa17d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:37.028 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:36 smithi003 bash[101665]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9 2024-09-12T02:28:37.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:36.885+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:37.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:36 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:36.885+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:37.528 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101692]: 2024-09-12 02:28:37.078860812 +0000 UTC m=+0.614708326 container remove 12d4436d5b01b2328118320746c028fafb50eb62cd3635f5caa475a818aa17d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-12T02:28:37.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:37 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:37.445+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:37.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:37 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:37.445+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:37.798 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.432190866 +0000 UTC m=+0.016183663 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:37.798 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.529536432 +0000 UTC m=+0.113529217 container create d2ee5762251ee3ab27fb6ffb3c945bd6b3dce951bc2c31efe253f63dde9c2f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T02:28:37.798 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.79869066 +0000 UTC m=+0.382683447 container init d2ee5762251ee3ab27fb6ffb3c945bd6b3dce951bc2c31efe253f63dde9c2f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T02:28:38.278 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.802328857 +0000 UTC m=+0.386321642 container start d2ee5762251ee3ab27fb6ffb3c945bd6b3dce951bc2c31efe253f63dde9c2f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T02:28:38.278 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.873681003 +0000 UTC m=+0.457673791 container attach d2ee5762251ee3ab27fb6ffb3c945bd6b3dce951bc2c31efe253f63dde9c2f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:38.278 INFO:journalctl@ceph.osd.9.smithi003.stdout:Sep 12 02:28:37 smithi003 podman[101786]: 2024-09-12 02:28:37.999233156 +0000 UTC m=+0.583225954 container died d2ee5762251ee3ab27fb6ffb3c945bd6b3dce951bc2c31efe253f63dde9c2f07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T02:28:38.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:37.883+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:38.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:37 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:37.883+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:38.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T02:28:38.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:38.624 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T02:28:38.624 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T02:28:38.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.10 2024-09-12T02:28:38.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:38.439+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:38.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:38.439+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:38.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 systemd[1]: Stopping Ceph osd.10 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:39.265 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:38.935+0000 7f09c3c04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:39.265 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:38.935+0000 7f09c3c04640 -1 osd.10 86 *** Got signal Terminated *** 2024-09-12T02:28:39.265 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:38 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:38.935+0000 7f09c3c04640 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:39.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:38.877+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:39.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:38.877+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:39.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:39.461+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:39.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:39 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:39.461+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:40.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:39 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:39.872+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:40.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:39 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:39.872+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:40.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:40.415+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:40.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:40.415+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:40.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:40 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:40.415+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:34.763344+0000 front 2024-09-12T02:27:34.763221+0000 (oldest deadline 2024-09-12T02:28:40.063284+0000) 2024-09-12T02:28:41.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:40 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:40.880+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:41.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:40 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:40.880+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:41.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:41 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:41.399+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:41.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:41 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:41.399+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:41.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:41 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:41.399+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:34.763344+0000 front 2024-09-12T02:27:34.763221+0000 (oldest deadline 2024-09-12T02:28:40.063284+0000) 2024-09-12T02:28:42.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:41.924+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:42.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:41 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:41.924+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:42.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:42 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:42.421+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:42.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:42 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:42.421+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:42.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:42 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:42.421+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:34.763344+0000 front 2024-09-12T02:27:34.763221+0000 (oldest deadline 2024-09-12T02:28:40.063284+0000) 2024-09-12T02:28:43.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:42 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:42.968+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:43.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:42 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:42.968+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:43.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:43 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:43.456+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.160392+0000 front 2024-09-12T02:27:22.160279+0000 (oldest deadline 2024-09-12T02:28:24.460216+0000) 2024-09-12T02:28:43.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:43 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:43.456+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.962133+0000 front 2024-09-12T02:27:28.962110+0000 (oldest deadline 2024-09-12T02:28:31.862246+0000) 2024-09-12T02:28:43.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:43 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10[60452]: 2024-09-12T02:28:43.456+0000 7f09c021a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:34.763344+0000 front 2024-09-12T02:27:34.763221+0000 (oldest deadline 2024-09-12T02:28:40.063284+0000) 2024-09-12T02:28:44.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:43 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:43.950+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:44.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:43 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:43.950+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:44.694 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:44 smithi026 podman[70306]: 2024-09-12 02:28:44.413960799 +0000 UTC m=+5.591731137 container died 3a7492d4f1c5cc4f1d9018ebb52802f0c41c8af898dfb8103ae5b8e8135d5a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:28:45.015 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:44 smithi026 podman[70306]: 2024-09-12 02:28:44.785385455 +0000 UTC m=+5.963155835 container cleanup 3a7492d4f1c5cc4f1d9018ebb52802f0c41c8af898dfb8103ae5b8e8135d5a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-12T02:28:45.015 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:44 smithi026 bash[70306]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10 2024-09-12T02:28:45.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:44 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:44.912+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:45.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:44 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:44.912+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:45.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:44 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:44.912+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:45.382 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70322]: 2024-09-12 02:28:45.070191771 +0000 UTC m=+0.655514717 container remove 3a7492d4f1c5cc4f1d9018ebb52802f0c41c8af898dfb8103ae5b8e8135d5a73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:45.765 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.284957648 +0000 UTC m=+0.015992086 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:45.766 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.387622325 +0000 UTC m=+0.118656817 container create 758010668fa94bc7b92cfb0c16c17b582209da1ac737574ce1d83ac307fadeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T02:28:45.766 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.60664577 +0000 UTC m=+0.337680208 container init 758010668fa94bc7b92cfb0c16c17b582209da1ac737574ce1d83ac307fadeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:45.766 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.609837165 +0000 UTC m=+0.340871603 container start 758010668fa94bc7b92cfb0c16c17b582209da1ac737574ce1d83ac307fadeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T02:28:46.092 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.840159457 +0000 UTC m=+0.571193977 container attach 758010668fa94bc7b92cfb0c16c17b582209da1ac737574ce1d83ac307fadeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:46.092 INFO:journalctl@ceph.osd.10.smithi026.stdout:Sep 12 02:28:45 smithi026 podman[70408]: 2024-09-12 02:28:45.840963544 +0000 UTC m=+0.571997992 container died 758010668fa94bc7b92cfb0c16c17b582209da1ac737574ce1d83ac307fadeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:28:46.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:45 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:45.892+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:46.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:45 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:45.892+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:46.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:45 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:45.892+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:46.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T02:28:46.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:46.436 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T02:28:46.436 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T02:28:46.436 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.11 2024-09-12T02:28:46.751 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 systemd[1]: Stopping Ceph osd.11 for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:47.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.750+0000 7f2e1d58a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:47.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.750+0000 7f2e1d58a640 -1 osd.11 86 *** Got signal Terminated *** 2024-09-12T02:28:47.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.750+0000 7f2e1d58a640 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T02:28:47.040 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.935+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:47.040 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.935+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:47.040 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:46 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:46.935+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:48.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:47 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:47.980+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:48.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:47 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:47.980+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:48.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:47 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:47.980+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:49.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:49 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:49.021+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:49.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:49 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:49.021+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:49.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:49 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:49.021+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:50.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:50 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:50.017+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:50.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:50 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:50.017+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:50.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:50 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:50.017+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:51.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:51 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:51.020+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-09-12T02:27:22.389993+0000 front 2024-09-12T02:27:22.390124+0000 (oldest deadline 2024-09-12T02:28:23.490027+0000) 2024-09-12T02:28:51.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:51 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:51.020+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2024-09-12T02:27:28.791143+0000 front 2024-09-12T02:27:28.791158+0000 (oldest deadline 2024-09-12T02:28:36.391392+0000) 2024-09-12T02:28:51.289 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:51 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11[61582]: 2024-09-12T02:28:51.020+0000 7f2e19ba0640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.92:6804 osd.2 since back 2024-09-12T02:27:38.692302+0000 front 2024-09-12T02:27:38.692192+0000 (oldest deadline 2024-09-12T02:28:44.592282+0000) 2024-09-12T02:28:52.684 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:52 smithi092 podman[71329]: 2024-09-12 02:28:52.410246492 +0000 UTC m=+5.785307032 container died 9cbf35ed1f2e4749e77f4ca037d6012289399ac9c7af1546db12f3f5c99eb0cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:28:53.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:52 smithi092 podman[71329]: 2024-09-12 02:28:52.80834131 +0000 UTC m=+6.183401893 container cleanup 9cbf35ed1f2e4749e77f4ca037d6012289399ac9c7af1546db12f3f5c99eb0cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T02:28:53.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:52 smithi092 bash[71329]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11 2024-09-12T02:28:53.416 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71345]: 2024-09-12 02:28:53.084834884 +0000 UTC m=+0.674501217 container remove 9cbf35ed1f2e4749e77f4ca037d6012289399ac9c7af1546db12f3f5c99eb0cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T02:28:53.721 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.319484806 +0000 UTC m=+0.016241175 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:28:53.721 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.427123198 +0000 UTC m=+0.123879556 container create 21620bc29255c404a4d9d27ccef6319752ad89681b5d6829c343a40a8a4018fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T02:28:53.722 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.638257851 +0000 UTC m=+0.335014208 container init 21620bc29255c404a4d9d27ccef6319752ad89681b5d6829c343a40a8a4018fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T02:28:53.722 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.641464813 +0000 UTC m=+0.338221171 container start 21620bc29255c404a4d9d27ccef6319752ad89681b5d6829c343a40a8a4018fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T02:28:53.722 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.721440424 +0000 UTC m=+0.418196794 container attach 21620bc29255c404a4d9d27ccef6319752ad89681b5d6829c343a40a8a4018fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T02:28:54.039 INFO:journalctl@ceph.osd.11.smithi092.stdout:Sep 12 02:28:53 smithi092 podman[71433]: 2024-09-12 02:28:53.828307741 +0000 UTC m=+0.525064109 container died 21620bc29255c404a4d9d27ccef6319752ad89681b5d6829c343a40a8a4018fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T02:28:54.443 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T02:28:54.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:54.489 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T02:28:54.489 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T02:28:54.489 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.a 2024-09-12T02:28:54.778 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:54 smithi003 systemd[1]: Stopping Ceph mds.a for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:55.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:54 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: 2024-09-12T02:28:54.797+0000 7f91bdf81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:55.278 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:54 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a[72690]: 2024-09-12T02:28:54.797+0000 7f91bdf81640 -1 mds.a *** got signal Terminated *** 2024-09-12T02:28:57.380 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:57 smithi003 podman[101889]: 2024-09-12 02:28:57.116115198 +0000 UTC m=+2.427150238 container died 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T02:28:57.738 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:57 smithi003 podman[101889]: 2024-09-12 02:28:57.561758665 +0000 UTC m=+2.872793698 container cleanup 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T02:28:57.738 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:57 smithi003 bash[101889]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a 2024-09-12T02:28:57.989 INFO:journalctl@ceph.mds.a.smithi003.stdout:Sep 12 02:28:57 smithi003 podman[101901]: 2024-09-12 02:28:57.738091794 +0000 UTC m=+0.614214639 container remove 665495a7f7fac786b3e54f6dcb20ca462df768e720dab49b3525ea1c39526517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T02:28:58.001 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T02:28:58.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:28:58.037 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T02:28:58.037 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T02:28:58.037 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.d 2024-09-12T02:28:58.278 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:28:58 smithi003 systemd[1]: Stopping Ceph mds.d for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:28:58.778 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:28:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d[71487]: 2024-09-12T02:28:58.382+0000 7f6ef0e63640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:28:58.778 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:28:58 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d[71487]: 2024-09-12T02:28:58.382+0000 7f6ef0e63640 -1 mds.d *** got signal Terminated *** 2024-09-12T02:29:01.093 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:29:00 smithi003 podman[102012]: 2024-09-12 02:29:00.80578213 +0000 UTC m=+2.546256038 container died 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T02:29:01.450 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:29:01 smithi003 podman[102012]: 2024-09-12 02:29:01.199784631 +0000 UTC m=+2.940258570 container cleanup 8dc3a71df0c646fc039e9d3fe23a4109edec79eb99702d63203b9d712bf247b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:29:01.450 INFO:journalctl@ceph.mds.d.smithi003.stdout:Sep 12 02:29:01 smithi003 bash[102012]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-d 2024-09-12T02:29:01.590 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T02:29:01.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:01.624 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T02:29:01.624 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T02:29:01.624 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.g 2024-09-12T02:29:02.028 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 12 02:29:01 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-g[72367]: 2024-09-12T02:29:01.945+0000 7f7634173640 -1 mds.g *** got signal Terminated *** 2024-09-12T02:29:05.278 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 12 02:29:05 smithi003 podman[102129]: 2024-09-12 02:29:05.114503012 +0000 UTC m=+3.261126175 container died 0ab59059144a053469c2969b1daf5432881d75676ccce94471a9af58422edc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T02:29:05.737 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 12 02:29:05 smithi003 podman[102129]: 2024-09-12 02:29:05.46150921 +0000 UTC m=+3.608132372 container cleanup 0ab59059144a053469c2969b1daf5432881d75676ccce94471a9af58422edc07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T02:29:05.738 INFO:journalctl@ceph.mds.g.smithi003.stdout:Sep 12 02:29:05 smithi003 bash[102129]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-g 2024-09-12T02:29:05.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T02:29:05.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:05.927 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T02:29:05.927 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T02:29:05.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.j 2024-09-12T02:29:06.249 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:06 smithi003 systemd[1]: Stopping Ceph mds.j for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:06.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:06 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j[71089]: 2024-09-12T02:29:06.249+0000 7f497fac5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:06.528 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:06 smithi003 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j[71089]: 2024-09-12T02:29:06.249+0000 7f497fac5640 -1 mds.j *** got signal Terminated *** 2024-09-12T02:29:09.591 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:09 smithi003 podman[102260]: 2024-09-12 02:29:09.318514075 +0000 UTC m=+3.169400678 container died 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T02:29:09.950 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:09 smithi003 podman[102260]: 2024-09-12 02:29:09.690895963 +0000 UTC m=+3.541782558 container cleanup 0b6d6896df3e945e3b14dc285f515dfd86caab4e2e1edf18b48460d1d27ee141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-12T02:29:09.950 INFO:journalctl@ceph.mds.j.smithi003.stdout:Sep 12 02:29:09 smithi003 bash[102260]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-j 2024-09-12T02:29:10.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T02:29:10.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:10.115 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T02:29:10.115 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T02:29:10.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.b 2024-09-12T02:29:10.410 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:10 smithi026 systemd[1]: Stopping Ceph mds.b for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:10.765 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:10 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: 2024-09-12T02:29:10.409+0000 7fc898249640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:10.765 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:10 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b[65647]: 2024-09-12T02:29:10.409+0000 7fc898249640 -1 mds.b *** got signal Terminated *** 2024-09-12T02:29:13.427 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:13 smithi026 podman[70511]: 2024-09-12 02:29:13.060568058 +0000 UTC m=+2.749997922 container died f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T02:29:13.765 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:13 smithi026 podman[70511]: 2024-09-12 02:29:13.518125672 +0000 UTC m=+3.207555527 container cleanup f7ebb6aae9dc9c5264933a0823beb54cd4ef78a7593c2a0c662cac59d66f4ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T02:29:13.765 INFO:journalctl@ceph.mds.b.smithi026.stdout:Sep 12 02:29:13 smithi026 bash[70511]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-b 2024-09-12T02:29:13.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T02:29:13.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:13.954 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T02:29:13.954 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T02:29:13.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.e 2024-09-12T02:29:14.515 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:14 smithi026 systemd[1]: Stopping Ceph mds.e for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:14.515 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:14 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e[65034]: 2024-09-12T02:29:14.280+0000 7f9e05c51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:14.515 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:14 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e[65034]: 2024-09-12T02:29:14.280+0000 7f9e05c51640 -1 mds.e *** got signal Terminated *** 2024-09-12T02:29:17.371 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:17 smithi026 podman[70645]: 2024-09-12 02:29:17.057384035 +0000 UTC m=+2.881998474 container died db4d59d415a531fcd8b12e3fe85cb1bd15fefbec1592a8d681a8d69fb38cdff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T02:29:17.371 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:17 smithi026 podman[70645]: 2024-09-12 02:29:17.371463253 +0000 UTC m=+3.196077693 container cleanup db4d59d415a531fcd8b12e3fe85cb1bd15fefbec1592a8d681a8d69fb38cdff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:29:17.720 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:17 smithi026 bash[70645]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e 2024-09-12T02:29:17.720 INFO:journalctl@ceph.mds.e.smithi026.stdout:Sep 12 02:29:17 smithi026 podman[70657]: 2024-09-12 02:29:17.598044028 +0000 UTC m=+0.534341718 container remove db4d59d415a531fcd8b12e3fe85cb1bd15fefbec1592a8d681a8d69fb38cdff3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T02:29:17.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T02:29:17.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:17.788 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T02:29:17.788 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T02:29:17.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.h 2024-09-12T02:29:18.015 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:17 smithi026 systemd[1]: Stopping Ceph mds.h for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:18.515 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:18 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: 2024-09-12T02:29:18.116+0000 7fb4159a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:18.515 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:18 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h[65334]: 2024-09-12T02:29:18.116+0000 7fb4159a1640 -1 mds.h *** got signal Terminated *** 2024-09-12T02:29:21.327 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 podman[70767]: 2024-09-12 02:29:21.058429365 +0000 UTC m=+3.035436457 container died 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T02:29:21.584 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 podman[70767]: 2024-09-12 02:29:21.400216556 +0000 UTC m=+3.377223679 container cleanup 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T02:29:21.585 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 bash[70767]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h 2024-09-12T02:29:21.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T02:29:21.848 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 podman[70780]: 2024-09-12 02:29:21.584636704 +0000 UTC m=+0.515533071 container remove 59781e152b2e43ce4e0792419655e39495fe825ea87acd8d7ea2ef4b1610537a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T02:29:21.848 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-12T02:29:21.849 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 systemd[1]: Stopped Ceph mds.h for c595fa10-70ab-11ef-bcea-c7b262605968. 2024-09-12T02:29:21.849 INFO:journalctl@ceph.mds.h.smithi026.stdout:Sep 12 02:29:21 smithi026 systemd[1]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.h.service: Consumed 2.579s CPU time. 2024-09-12T02:29:21.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:21.859 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T02:29:21.860 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T02:29:21.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.k 2024-09-12T02:29:22.179 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:21 smithi026 systemd[1]: Stopping Ceph mds.k for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:22.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:22 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: 2024-09-12T02:29:22.178+0000 7f1745358640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:22.515 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:22 smithi026 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k[64720]: 2024-09-12T02:29:22.178+0000 7f1745358640 -1 mds.k *** got signal Terminated *** 2024-09-12T02:29:25.346 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:25 smithi026 podman[70889]: 2024-09-12 02:29:25.05823366 +0000 UTC m=+2.976256968 container died 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:29:25.688 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:25 smithi026 podman[70889]: 2024-09-12 02:29:25.437023743 +0000 UTC m=+3.355047052 container cleanup 61457315352cbbbf844faff87f9fde0fdd14d424e1b8ed6aa4921ef19bb030a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T02:29:25.688 INFO:journalctl@ceph.mds.k.smithi026.stdout:Sep 12 02:29:25 smithi026 bash[70889]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-k 2024-09-12T02:29:25.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T02:29:25.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:25.861 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T02:29:25.861 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T02:29:25.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.c 2024-09-12T02:29:26.207 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:25 smithi092 systemd[1]: Stopping Ceph mds.c for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:26.471 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:26 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: 2024-09-12T02:29:26.206+0000 7f74b0d4a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:26.471 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:26 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c[66544]: 2024-09-12T02:29:26.206+0000 7f74b0d4a640 -1 mds.c *** got signal Terminated *** 2024-09-12T02:29:29.307 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:29 smithi092 podman[71550]: 2024-09-12 02:29:29.045260251 +0000 UTC m=+2.985276599 container died 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T02:29:29.690 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:29 smithi092 podman[71550]: 2024-09-12 02:29:29.4058484 +0000 UTC m=+3.345864758 container cleanup 3eee5212662c7a9d4675c2cb8331ea1b2f5e1e962273b3261b8aad7e7f446620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T02:29:29.690 INFO:journalctl@ceph.mds.c.smithi092.stdout:Sep 12 02:29:29 smithi092 bash[71550]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-c 2024-09-12T02:29:29.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T02:29:29.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:29.872 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T02:29:29.872 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T02:29:29.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.f 2024-09-12T02:29:30.289 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:29 smithi092 systemd[1]: Stopping Ceph mds.f for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:30.290 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:30 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: 2024-09-12T02:29:30.184+0000 7f2eaf15a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:30.290 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:30 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f[66245]: 2024-09-12T02:29:30.184+0000 7f2eaf15a640 -1 mds.f *** got signal Terminated *** 2024-09-12T02:29:33.311 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:33 smithi092 podman[71683]: 2024-09-12 02:29:33.04191749 +0000 UTC m=+2.956733384 container died fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T02:29:33.685 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:33 smithi092 podman[71683]: 2024-09-12 02:29:33.40924036 +0000 UTC m=+3.324056253 container cleanup fa5e311a6b5966903732cd7349637cdbfed68b4869f770f74fbc7b8da9985565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T02:29:33.686 INFO:journalctl@ceph.mds.f.smithi092.stdout:Sep 12 02:29:33 smithi092 bash[71683]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-f 2024-09-12T02:29:33.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T02:29:33.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:33.864 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T02:29:33.864 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T02:29:33.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.i 2024-09-12T02:29:34.289 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:33 smithi092 systemd[1]: Stopping Ceph mds.i for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:34.289 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:29:34.179+0000 7f373f50d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:34.289 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:34 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i[66852]: 2024-09-12T02:29:34.179+0000 7f373f50d640 -1 mds.i *** got signal Terminated *** 2024-09-12T02:29:37.289 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:37 smithi092 podman[71805]: 2024-09-12 02:29:37.038595811 +0000 UTC m=+2.961767686 container died ed0c300e104592aba7fbbd0ff31be0b59c397cd990edf3d2c389a23d821f7c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T02:29:37.681 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:37 smithi092 podman[71805]: 2024-09-12 02:29:37.460766625 +0000 UTC m=+3.383938498 container cleanup ed0c300e104592aba7fbbd0ff31be0b59c397cd990edf3d2c389a23d821f7c5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T02:29:37.681 INFO:journalctl@ceph.mds.i.smithi092.stdout:Sep 12 02:29:37 smithi092 bash[71805]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-i 2024-09-12T02:29:37.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T02:29:37.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:37.919 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T02:29:37.919 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T02:29:37.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.l 2024-09-12T02:29:38.242 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:38 smithi092 systemd[1]: Stopping Ceph mds.l for c595fa10-70ab-11ef-bcea-c7b262605968... 2024-09-12T02:29:38.539 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: 2024-09-12T02:29:38.241+0000 7f5296434640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:29:38.539 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:38 smithi092 ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l[65941]: 2024-09-12T02:29:38.241+0000 7f5296434640 -1 mds.l *** got signal Terminated *** 2024-09-12T02:29:41.313 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:41 smithi092 podman[71928]: 2024-09-12 02:29:41.049136243 +0000 UTC m=+2.911678758 container died 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T02:29:41.584 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:41 smithi092 podman[71928]: 2024-09-12 02:29:41.391590872 +0000 UTC m=+3.254133379 container cleanup 497530ec060b9cb2f6e33eadd368525de49ddea405810beacfeffd597d005e2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T02:29:41.584 INFO:journalctl@ceph.mds.l.smithi092.stdout:Sep 12 02:29:41 smithi092 bash[71928]: ceph-c595fa10-70ab-11ef-bcea-c7b262605968-mds-l 2024-09-12T02:29:41.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c595fa10-70ab-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T02:29:41.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T02:29:41.738 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T02:29:41.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:29:41.950 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:29:44.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:29:44.837 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:29:47.441 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T02:29:47.639 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:29:50.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:29:50.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:29:50.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:29:50.196 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T02:29:50.198 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi003/crash 2024-09-12T02:29:50.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:29:50.235 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:29:50.235 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:29:50.237 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi026/crash 2024-09-12T02:29:50.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:29:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:29:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:29:50.280 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi092/crash 2024-09-12T02:29:50.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash -- . 2024-09-12T02:29:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T02:29:50.315 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-09-12T02:29:50.317 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T02:29:50.317 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T02:29:50.406 INFO:teuthology.orchestra.run.smithi003.stdout:2024-09-12T02:19:53.404264+0000 mds.f (mds.2) 2 : cluster [WRN] Scrub error on inode 0x10000000279 (/volumes/qa/sv_1/bec4a6cd-04f3-4397-b5fa-8e22c4a0b372/client.0/tmp/ffsb/.git/objects/9a/902847fb04ba0435f838cf7cdb9b56c42fb61b) see mds.f log and `damage ls` output for details 2024-09-12T02:29:50.407 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-12T02:29:50.407 INFO:tasks.cephadm:Compressing logs... 2024-09-12T02:29:50.407 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:29:50.410 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:29:50.413 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T02:29:50.439 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T02:29:50.439 INFO:teuthology.orchestra.run.smithi026.stderr:: No such file or directory 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T02:29:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T02:29:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:29:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:29:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T02:29:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:29:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T02:29:50.445 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T02:29:50.445 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T02:29:50.445 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log: --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T02:29:50.446 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:29:50.449 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T02:29:50.449 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T02:29:50.449 INFO:teuthology.orchestra.run.smithi092.stderr:gzip/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.11.log: -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T02:29:50.450 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:29:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T02:29:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T02:29:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T02:29:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T02:29:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T02:29:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T02:29:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T02:29:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T02:29:50.454 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 86.2% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr: 94.6% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T02:29:50.455 INFO:teuthology.orchestra.run.smithi003.stderr: 94.6% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T02:29:50.471 INFO:teuthology.orchestra.run.smithi003.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:29:50.506 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 90.4% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:29:50.519 INFO:teuthology.orchestra.run.smithi092.stderr: 92.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T02:29:50.554 INFO:teuthology.orchestra.run.smithi003.stderr: 82.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:29:50.598 INFO:teuthology.orchestra.run.smithi092.stderr: 82.6% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:29:50.621 INFO:teuthology.orchestra.run.smithi092.stderr: 86.3% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:29:50.629 INFO:teuthology.orchestra.run.smithi092.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:29:50.684 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:29:50.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T02:29:50.706 INFO:teuthology.orchestra.run.smithi026.stderr: 82.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T02:29:50.740 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:29:50.748 INFO:teuthology.orchestra.run.smithi026.stderr: 90.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:29:50.766 INFO:teuthology.orchestra.run.smithi092.stderr: 94.4% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T02:29:50.797 INFO:teuthology.orchestra.run.smithi003.stderr: 93.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T02:29:50.799 INFO:teuthology.orchestra.run.smithi026.stderr: 86.3% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T02:29:50.801 INFO:teuthology.orchestra.run.smithi026.stderr: 90.4% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T02:29:50.880 INFO:teuthology.orchestra.run.smithi092.stderr: 90.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T02:29:50.924 INFO:teuthology.orchestra.run.smithi026.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T02:29:50.928 INFO:teuthology.orchestra.run.smithi092.stderr: 93.3% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T02:29:51.280 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T02:29:51.404 INFO:teuthology.orchestra.run.smithi003.stderr: 93.4% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T02:29:51.734 INFO:teuthology.orchestra.run.smithi003.stderr: 90.1% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T02:29:51.739 INFO:teuthology.orchestra.run.smithi026.stderr: 93.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T02:29:52.419 INFO:teuthology.orchestra.run.smithi026.stderr: 94.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T02:29:53.494 INFO:teuthology.orchestra.run.smithi092.stderr: 91.6% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T02:29:55.940 INFO:teuthology.orchestra.run.smithi026.stderr: 91.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T02:29:56.913 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T02:30:04.449 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T02:30:30.327 INFO:teuthology.orchestra.run.smithi026.stderr: 96.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T02:32:34.356 INFO:teuthology.orchestra.run.smithi003.stderr: 92.0% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T02:35:44.686 INFO:teuthology.orchestra.run.smithi092.stderr: 95.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T02:36:19.611 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T02:39:18.338 INFO:teuthology.orchestra.run.smithi003.stderr: 91.7% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T02:40:15.239 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T02:42:19.792 INFO:teuthology.orchestra.run.smithi003.stderr: 91.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T02:42:48.385 INFO:teuthology.orchestra.run.smithi003.stderr: 91.8% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T02:42:48.387 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-09-12T02:42:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:real 12m57.962s 2024-09-12T02:42:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:user 7m45.486s 2024-09-12T02:42:48.387 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m18.158s 2024-09-12T02:43:32.259 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T02:44:04.865 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T02:44:04.867 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-09-12T02:44:04.867 INFO:teuthology.orchestra.run.smithi092.stderr:real 14m14.440s 2024-09-12T02:44:04.867 INFO:teuthology.orchestra.run.smithi092.stderr:user 7m29.669s 2024-09-12T02:44:04.867 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m19.230s 2024-09-12T02:45:00.076 INFO:teuthology.orchestra.run.smithi026.stderr: 92.0% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T02:45:05.227 INFO:teuthology.orchestra.run.smithi026.stderr: 92.0% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T02:47:49.077 INFO:teuthology.orchestra.run.smithi026.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T02:47:54.188 INFO:teuthology.orchestra.run.smithi026.stderr: 91.9% -- replaced with /var/log/ceph/c595fa10-70ab-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T02:47:54.191 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-12T02:47:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:real 18m3.763s 2024-09-12T02:47:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:user 8m0.023s 2024-09-12T02:47:54.191 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m20.299s 2024-09-12T02:47:54.192 INFO:tasks.cephadm:Archiving logs... 2024-09-12T02:47:54.193 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi003/log 2024-09-12T02:47:54.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:49:07.066 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi026/log 2024-09-12T02:49:07.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:50:21.788 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi092/log 2024-09-12T02:50:21.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T02:51:36.948 INFO:tasks.cephadm:Removing cluster... 2024-09-12T02:51:37.006 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force 2024-09-12T02:51:37.235 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:51:38.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force 2024-09-12T02:51:38.364 INFO:teuthology.orchestra.run.smithi026.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:51:39.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c595fa10-70ab-11ef-bcea-c7b262605968 --force 2024-09-12T02:51:39.477 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c595fa10-70ab-11ef-bcea-c7b262605968 2024-09-12T02:51:40.232 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T02:51:40.232 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:40.252 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:40.284 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T02:51:40.309 INFO:tasks.cephadm:Teardown complete 2024-09-12T02:51:40.310 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T02:51:40.327 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T02:51:40.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:51:40.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:51:40.333 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T02:51:40.465 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T02:51:40.479 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T02:51:40.479 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:40.482 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:40.485 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:51:40.497 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:40.499 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:51:40.523 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:40.523 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-09-12T02:51:40.524 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 7 377 109 -3233us[-3281us] +/- 50ms 2024-09-12T02:51:40.524 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 180 +1591us[+1547us] +/- 59ms 2024-09-12T02:51:40.524 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +3235us[+3235us] +/- 88ms 2024-09-12T02:51:40.524 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -3212us[-3191us] +/- 51ms 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +1658us[+1679us] +/- 61ms 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +3305us[+3325us] +/- 88ms 2024-09-12T02:51:40.532 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 -3199us[-3264us] +/- 51ms 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 +1690us[+1690us] +/- 61ms 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +3317us[+3253us] +/- 88ms 2024-09-12T02:51:40.555 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:51:40.556 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T02:51:40.573 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T02:51:40.573 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T02:51:40.593 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:51:40.710 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:51:40.828 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T02:51:40.951 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:51:40.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:40.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:51:41.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:41.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:51:41.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:41.043 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T02:51:41.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T02:51:41.075 INFO:teuthology.task.internal:Duration was 3021.246782 seconds 2024-09-12T02:51:41.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T02:51:41.098 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T02:51:41.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:51:41.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:51:41.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T02:51:41.160 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:51:41.166 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:51:41.193 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:51:41.592 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T02:51:41.592 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-09-12T02:51:41.592 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:51:41.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-09-12T02:51:41.620 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:51:41.655 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-09-12T02:51:41.655 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T02:51:41.700 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T02:51:41.700 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:51:41.703 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:51:41.706 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T02:51:41.744 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T02:51:41.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:51:41.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:51:41.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T02:51:42.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T02:51:42.600 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T02:51:42.600 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:51:42.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:51:42.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T02:51:42.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T02:51:42.731 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.762 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-09-12T02:51:42.762 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-09-12T02:51:42.767 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-09-12T02:51:42.790 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:42.832 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:42.849 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T02:51:42.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:51:42.864 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T02:51:42.877 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T02:51:42.879 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi003 2024-09-12T02:51:42.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:51:42.931 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi026 2024-09-12T02:51:42.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:51:42.970 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901597/remote/smithi092 2024-09-12T02:51:42.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T02:51:43.007 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T02:51:43.008 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:51:43.010 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:51:43.012 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T02:51:43.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T02:51:43.070 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T02:51:43.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T02:51:43.111 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T02:51:43.111 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:51:43.114 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:51:43.116 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T02:51:43.126 INFO:teuthology.orchestra.run.smithi003.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:51 /home/ubuntu/cephtest 2024-09-12T02:51:43.129 INFO:teuthology.orchestra.run.smithi026.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:51 /home/ubuntu/cephtest 2024-09-12T02:51:43.131 INFO:teuthology.orchestra.run.smithi092.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 02:51 /home/ubuntu/cephtest 2024-09-12T02:51:43.138 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T02:51:43.149 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T02:51:43.288 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 3021.246782064438 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-12T02:51:43.289 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:51:43.381 INFO:teuthology.run:FAIL